JP2023167720A - Dialogue device, dialogue system, and dialogue method - Google Patents

Abstract

To provide a dialogue system capable of returning an appropriate response suitable for each category relative to a harmful input of various categories.SOLUTION: A dialogue system includes an information processing device 205 and a server 201 connected to the information processing device via a network 202. The server includes: an input unit 401 for receiving input data; a plurality of dialogue models 408 that is trained so as to output a safe response sentence to harmful input data for each attack target by the input data to each generate a response sentence to the input data; and a response selection unit 405 for selecting and outputting the response sentence from a plurality of response sentences based on a predetermined reference.SELECTED DRAWING: Figure 4A

Description

The present invention relates to a dialogue device, a dialogue system, and a dialogue method.

In a dialogue system created by collecting SNS and chat dialogue logs from the Web, there is a technique described in, for example, Japanese Patent Laid-Open No. 2006-11592 (Patent Document 1) to prevent inappropriate responses such as slander. . The purpose of this publication is to prevent sentences containing prohibited words from being output. Therefore, the dividing unit 81 divides the input sentence into words. The word list storage unit 83 stores words that are permitted to be used in a sentence. If all the words in the sentence supplied from the dividing unit 81 are words that are allowed to be used in the sentence stored in the word list storage unit 83, the sentence deletion unit 82 outputs the sentence. Patent Document 1 describes, for example, a technique applied to selecting (output examples of) input/output pairs used for creating a response sentence for a dialogue of a robot device that performs a dialogue.

Furthermore, in Non-Patent Document 1, in a dialogue system using a pre-trained language model, dialogue logs of utterances containing slander and appropriate responses are collected from SNS, and this dialogue log is used to evaluate the language A technology is described that outputs an appropriate response to slanderous utterances by additionally learning a model.

Japanese Patent Application Publication No. 2006-11592

Ashutosh Baheti and 2 others “Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts”, Georgia Institute of Technology, Atlanta, GA, USA [online]＜URL: https://arxiv.org/abs/2108.11830＞

In Patent Document 1, it is determined whether the response can be outputted, that is, whether it is inappropriate or not, based only on the words included in the response, and it is not possible to suppress responses that are inappropriate in meaning even if they do not include harmful words. . For example, if you respond to slander with "I think so too," it will be an inappropriate response, but the response does not contain any harmful words, and the words alone cannot be judged as inappropriate. Can not.

Non-Patent Document 1 solves this problem, but since there are various categories of harmful speech, such as racial/ethnic discrimination and gender discrimination, it treats these as one type of harmful speech. , it may not be possible to output an appropriate response according to the category due to harmful comments.

Accordingly, the present invention provides an interaction system that can respond to harmful inputs of various categories and return appropriate responses suitable for each category.

In order to achieve the above object, for example, the configurations described in the claims are adopted.

The present application includes a plurality of means for solving the above problems, and one example thereof is an interaction system having an information processing device and a server connected to the information processing device via a network. The server has an input section that receives input data including input text such as posted comments and reply comments to the posted comments from the information processing device, and a safe response text for each attack target by the input data against harmful input data. A single response sentence is generated based on a predetermined standard from multiple dialogue models that are trained to output and each generates a response sentence in response to input data, and multiple response sentences generated from the multiple dialogue models. This is a dialogue system including a response selection unit that selects and outputs a response.

According to the present invention, it is possible to provide a dialogue system that can respond to various types of harmful comments with appropriate responses according to their categories.

Problems, configurations, and effects other than those described above will be made clear by the following description of the embodiments.

FIG. 2 is an explanatory diagram of dialogue processing of the dialogue system according to the first embodiment. FIG. 2 is an explanatory diagram showing an example of a system configuration of a dialogue system. 1 is a block diagram illustrating an example of a hardware configuration of a dialogue system; FIG. FIG. 1 is a diagram illustrating an example of a functional configuration of a dialogue system according to a first embodiment. FIG. 2 is a diagram illustrating a picture of data stored in a storage device of the dialogue system according to the first embodiment. FIG. 2 is an explanatory diagram showing a method for formatting a dialogue log of a bulletin board type SNS. FIG. 3 is a diagram showing an example of annotation for a comment. FIG. 3 is a diagram illustrating a flowchart of a harmfulness evaluation model creation process of the dialogue system according to the first embodiment. FIG. 3 is an explanatory diagram of a data flow in a hazard evaluation model creation process of the dialogue system according to the first embodiment. FIG. 3 is a diagram illustrating a flowchart of harmful category compatible dialog model creation processing of the dialog system according to the first embodiment. FIG. 2 is an explanatory diagram of data flow in harmful category compatible dialogue model creation processing of the dialogue system according to the first embodiment. FIG. 3 is a diagram showing a flowchart of dialogue processing of the dialogue system according to the first embodiment. FIG. 2 is a block diagram illustrating an example of a functional configuration of a dialogue system according to a second embodiment. FIG. 7 is a diagram showing a flowchart of interaction processing of the interaction system according to the second embodiment. FIG. 7 is a diagram showing an example of a harmfulness score of the dialogue system according to the second embodiment. FIG. 7 is a diagram illustrating an example of a weighting coefficient for calculating a threshold of a harmfulness score by category and a comprehensive harmfulness score of the dialogue system according to the second embodiment. FIG. 7 is a block diagram illustrating an example of a functional configuration of a dialogue system according to a third embodiment. FIG. 7 is a diagram showing a flowchart of a harmfulness evaluation model creation process of the dialogue system according to the third embodiment. FIG. 7 is a diagram showing an example of registration of co-occurrence categories according to the third embodiment.

Embodiments of the present invention will be described below with reference to the drawings. The following description and drawings are examples for explaining the present invention, and are omitted and simplified as appropriate for clarity of explanation. The present invention can also be implemented in various other forms. Unless specifically limited, each component may be singular or plural.

The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

In addition, in the following explanation, processing performed by executing a program may be explained, but the program is executed by a processor (for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit)). The processor may be the main body of the processing in order to perform the processing using appropriate storage resources (for example, memory) and/or interface devices (for example, communication ports). Similarly, the subject of processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The main body of the processing performed by executing the program may be an arithmetic unit, and may include a dedicated circuit (for example, FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit)) that performs specific processing. .

The dialogue device, dialogue system, and dialogue method according to the present embodiment are applied to, for example, QA for personal computer operations, QA for insurance enrollment, reception robots, call centers, and robots for the elderly. In particular, reception robots, call centers, and robots for the elderly can output safe response sentences to input data that includes a wide variety of input sentences. It can provide value such as alleviating loneliness.

FIG. 1 shows an explanatory diagram of interaction processing according to the first embodiment of the present invention. The dialogue system 100 receives input data 110 including an input sentence from a user, and outputs a response sentence 111. The input data 110 includes an input sentence, and the input sentence is an example of a harmful remark, and includes a posted comment, a posted comment, and a reply comment to the posted comment. Hereinafter, the explanation will be continued assuming that the input data 110 is the input sentence 110. The dialogue system 100 includes a dialogue model 101 compatible with harmful category 1, which is adjusted to be able to appropriately respond to harmful input sentences classified as harmful category 1, and a dialogue model 101 that is adjusted to be able to appropriately respond to harmful input sentences classified as harmful category 2. It is equipped with a dialogue model 102 compatible with harmful category 2, which has been adjusted as shown in FIG. to generate a response sentence.

Here, the term "harmful category" refers to an "attack target" that is attacked by input data including posted comments, and input sentences included in input data including posted comments and reply comments to posted comments, and includes, for example, racial - Articles about ethnic discrimination or crimes. Therefore, the dialogue model 101 compatible with harmful category 1, the dialogue model 102 compatible with harmful category 2, and the dialogue model 103 compatible with harmful category N can also be referred to as the dialogue model compatible with attack targets.

Then, the dialogue system 100 evaluates each response sentence generated by the dialogue model to determine whether it is harmful or not, calculates a harm score, and selects and outputs the response sentence with the lowest harm score. In FIG. 1, a response sentence 104 output by the harmful category 1 compatible dialogue model 101 is determined to be the least harmful to an input sentence 110, and is output as a response sentence 111.

<Dialogue system>
FIG. 2 is an explanatory diagram showing an example of the system configuration of the dialogue system according to the first embodiment. The dialogue system 100 is, for example, a client server system, and includes a server 201 and an information processing device 205 such as a PC 203 or a smartphone 204. The server 201 and the information processing device 205 can communicate via a network 202 such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

In the case of a client-server system, the interaction program is installed on the server 201. Therefore, the server 201 executes response generation, model management, response selection, and dialog history management processing as a dialog system. In this case, the information processing device 205 serves as an interface for transmitting text to be processed to the server 201, receiving processing results from the server 201, and inputting text to be processed.

On the other hand, in the case of a stand-alone type, the dialog program is installed on the information processing device 205, and the server 201 is not required. Therefore, the information processing device 205 functions as an interaction device, inputting text, generating a response to the input text, and outputting the results.

<Example of hardware configuration of dialogue system>
FIG. 3 is a block diagram showing an example of the hardware configuration of the dialog device 300.

The dialogue device 300 corresponds to the server 201 in the case of a dialogue system, and corresponds to the information processing device 205 in the case of a stand-alone type. The dialog device 300 includes a processor 301 , a storage device 302 , an input device 303 , an output device 304 , and a communication interface (communication IF) 305 . The processor 301, storage device 302, input device 303, output device 304, and communication IF 305 are connected by a bus 306. Processor 301 controls interaction device 300 . The storage device 302 becomes a work area for the processor 301. Furthermore, the storage device 302 is a non-temporary or temporary recording medium that stores various programs and data. Examples of the storage device 302 include ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), and flash memory. Input device 303 inputs data. Examples of the input device 303 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a biosensor. Output device 304 outputs data. Examples of the output device 304 include a display, a printer, and a speaker. Communication IF 305 connects to network 202 and transmits and receives data.

<Functional configuration example of dialogue system 100>
FIG. 4A is a block diagram showing an example of the functional configuration of the dialogue system 100 according to the first embodiment. The dialogue system 100 is a computer in which a dialogue processing program is installed, and is an information processing device 205 or a server 201. The dialog system 100 uses an input section 401, a response generation section 404, a model management section 402, a dialog model group 408, a toxicity evaluation model 403, a response selection section 405, a dialog history management section 407, and an output section 406 using a dialog processing program. Realize.

As shown in FIG. 4B, an input unit 401, a model management unit 402, a response generation unit 404, a response selection unit 405, an output unit 406, and a dialog history management unit 407 are configured by a dialog processing program stored in a storage device 302 by a processor 301. By executing 410, various functions are realized.

Furthermore, the interaction model group 408, the toxicity evaluation model 403, and the base interaction model 1002 are also stored in the storage device 302.

Further, various data such as a harmful word list 802 and annotation data 600, which will be described later, are stored in various data 411 of the storage device 302.

The input unit 401 corresponds to the input device 303 in FIG. 3, and is a module that reads text data, which is an input sentence, and processes the text data and inputs it to the response generation unit 404. In addition, all text data are registered through the input unit 401 when creating the toxicity evaluation model 403 and the interaction model group 408.

The model management unit 402 manages the interaction model group 408 used by the response generation unit 404 and the toxicity evaluation model 403 used by the response selection unit 405. The model management unit 402 inputs text (input sentence) to a predetermined model according to instructions from the response generation unit 404, and returns the output result (response sentence) of the model. Furthermore, the model management unit 402 inputs the response sentence into a predetermined model according to instructions from the response selection unit 405, calculates a harmfulness score of the response sentence using the predetermined model, and grasps the harmfulness of the response sentence. Furthermore, the model management unit 402 also creates and registers various models.

The response generation unit 404 inputs the text (input sentence) received from the input unit 401 to the dialog model group 408 via the model management unit 402, and generates a dialog model corresponding to each harmful category (hereinafter simply referred to as a dialog model). Get the response text from. When text is input, a dialogue model generates a response sentence for that text. The dialogue model is a neural language model learned from multiple round-trip dialogues obtained from SNS and the like.

FIG. 5 is an explanatory diagram of the process of training a neural language model and creating a dialogue model using the texts of posted comments and reply comments acquired from the bulletin board type SNS input from the input unit 401.

500 is an example of a bulletin board type SNS thread (tree structure of posts and a series of replies) input from the input unit 401. Comments 502 and 505 are replies to the posted comment 501. Further, there is a reply comment 503 as a reply to the reply comment 502, and a reply comment 504 as a reply to the reply comment 503.

510 is data in which the input unit 401 formats 500 posted comments and reply comments for neural language model learning. 500 has two branches, 502 to 504 and 505 to 506, and in 510, the branches are formatted as 511 and 512, respectively. 511 is a text that connects comments 501 to 504, and "|" is a special token (word) that indicates the boundary between comments, and the neural language model is composed of multiple comments based on this token. Recognize the range of each comment from the text.

A dialogue model 522 is created by preparing a large amount of dialogue data in the format 510 including posted comments and reply comments as input sentences and learning the neural language model. It looks like it's going to snow.'', the output 523 ``It's the first snow.'' is obtained as a plausible response to this input. The dialogue models in the dialogue model group 408 are dialogue models that have been trained to output appropriate responses to harmful comments using the method described below.

A dialogue model using a neural language model can be created using, for example, GPT-2 (Generative Pre-trained Transformer 2). For example, DialoGPT is an interaction model that is trained using the GPT-2 architecture using data obtained from SNS. It should be noted that it is only necessary to input a text including a history of multiple round-trip conversations and output a response text, and it may be realized using other techniques. For example, another architecture is BART (Bidirectional Auto-Regressive Transformer).

The toxicity evaluation model 403 is a sentence classification model that determines whether the text input from the input unit 401 is harmful content such as harmful remarks, and is a sentence classification model that determines whether the input text (input sentence) is harmful or not. Perform binary classification. Furthermore, the probability of the classification result calculated by passing the output of the classification model through a softmax function is output as a harmfulness score.

The hazard evaluation model 403 is also realized by GPT-2 (Generative Pre-trained Transformer 2). The hazard evaluation model 403 prepares data in which a binary label of harmful content or safe content is assigned to each comment using the method described later, and adds a function that predicts the label of a sentence when the sentence is input. Learn.

The response selection unit 405 uses the toxicity evaluation model 403 to evaluate each response sentence output by the response generation unit 404 using the dialogue model via the model management unit 402. The response selection unit 405 outputs the response sentence with the lowest harmfulness score below a predetermined threshold to the output unit 406 and the dialogue history management unit 407. The output unit 406 outputs the response sentence selected by the response selection unit 405 to the output device 304.

The dialogue history management unit 407 holds the response sentence selected by the response selection unit 405 as a dialogue history, and outputs the held dialogue history to the response generation unit 404 when the input unit 401 receives an input. For example, the dialogue history management unit 407 manages the correspondence between the input posted comment 521 in FIG. 5 and the response sentence 523 selected by the response selection unit 405.

<Hazard assessment model creation process>
In the hazard evaluation model creation process, in the dialogue system 100 according to the first embodiment, the model management unit 402 creates a hazard evaluation model necessary for dialogue processing, and creates a hazard category corresponding dialogue model. This is the process of creating another hazard assessment model. The model management unit 402 annotates posted comments acquired from the bulletin board type SNS and reply comments thereto as to whether or not they are harmful via the user, and uses this annotation to learn a harmfulness evaluation model. FIG. 7 shows a flowchart of the process of creating a hazard evaluation model, and FIG. 8 shows the data flow in this process. FIG. 6 shows an example of data to be annotated.

As shown in FIG. 8, the model management unit 402 first extracts threads containing harmful comments from the SNS dump data 801, and formats the data for learning the harmfulness evaluation model (step S701). Dump data 801 is data that summarizes SNS logs for a certain period of time. If the dump data cannot be acquired, the data is acquired from the input unit 401 by crawling the SNS.

To extract threads containing harmful remarks, threads containing words in the harmful speech-related term list 802 are screened. The harmful word list 802 is a list of words that frequently appear in harmful comments such as slander, and is stored as various data 411 in the storage device 302. The word list may be created using a list of prohibited words published on the Web, or may be created independently by the user. The data formatting in step S701 is a process performed by the input unit 401 to format data in format 500 illustrated in FIG. 5 into data in format 510.

Next, the model management unit 402 performs annotation on the data 803 formatted in step S701 to add data regarding the presence or absence of harmfulness. Since the annotation is performed manually, the model management unit 402 outputs the data to be annotated to the output device 304 and receives user input through the input device 303.

The model management unit 402 extracts one piece of data from the formatted data 803 (step S702). Here, the data is data that is a combination of posted comments and their reply comments exemplified by 511 and 512 in FIG. In this way, by simultaneously annotating posted comments and their reply comments, the harmfulness is annotated not only based on the comment alone but also on previous comments. For example, comments that agree with harmful comments are also considered harmful.

Hazards are selected from predefined categories. "Harmful categories" are "attack targets" that are attacked by input data including posted comments, posted comments and input data such as reply comments to posted comments, and include, for example, race/ethnicity. , gender, religion, appearance, health (disability), politics/socioeconomics, etc. Furthermore, since there are some harmful comments that fall into multiple categories, such as those related to a specific race or gender, the harmfulness label should be able to specify multiple categories. In this way, all comments included in one data are annotated (step S703).

An example of annotation data is shown in FIG. FIG. 6 shows annotation data 600, in which a posted comment 601 and its reply comments 602 to 604 are labeled as harmful categories. As described above, the model management unit 402 collectively outputs a series of reply comments to the output device 304 and allows the user to perform annotation, thereby making it possible to assign a label in consideration of the context. 605 to 608 are labels given by annotation. The annotation data 600 is data that is labeled with a category that specifies an attack target for a comment.

Comment 601 "There are a lot of criminals in that country." is a racially/ethnically discriminatory comment, so it is assigned a "racial/ethnic" label. Comment 602 "That's not true." is a comment that opposes comment 601 and is not harmful, so it is given a label of "not harmful." Comment 603 "No, I think that's right." is not harmful on its own, but it can be judged as a racial/ethnic discriminatory comment that agrees with comment 601, so it is given the label "racial/ethnic". do. Comment 604, ``Men from that country are lazy criminals.'' contains the two harmful effects of racial/ethnic discrimination and gender discrimination. Assign one label.

When a user completes the annotation of one piece of data and receives a registration instruction from the user through the input device 303, the model management unit 402 registers the data and assigns a category to the data that has been annotated so far, including this data. Aggregate the number of data for each.

If there is a category in which the number of annotated data is less than the preset number of data, the model management unit 402 returns to step S702, extracts the next data, and continues annotation. In this way, annotation is performed until a sufficient number of data have been registered for each category, thereby preventing accuracy differences between categories in the hazard assessment model.

When a predetermined number of data are registered for all categories (step S704: YES), the model management unit 402 uses the annotated data 804 to train the hazard evaluation model 403 (step S705). Each piece of data is annotated with hazards including categories, but in this step, the model management unit 402 creates a hazard evaluation model that performs binary classification of whether or not it is harmful without dividing the data into categories. Therefore, the model management unit 402 trains the toxicity evaluation model 403 by converting the correct label into a binary value indicating whether or not it is harmful for the annotated data.

Next, the model management unit 402 classifies the annotated data 804 for each category in order to create a hazard evaluation model 403 for each category (step S706). After classifying the data, the model management unit 402 focuses on one of the categories (step S707), and uses the data 805 of the category to train a hazard evaluation model (step S708). If there is an unprocessed category (step S709: YES), the model management unit 402 returns to step S707 and learns the toxicity evaluation model for the category. When learning of the hazard evaluation models for all categories is completed in this manner (step S709), the hazard evaluation model creation process is completed.

<Harmful category compatible dialogue model creation process>
The harmful category compatible dialogue model (dialogue model) creation process is a process of creating a dialogue model that can appropriately respond to harmful comments for each target of attack, that is, for each harmful category. The model management unit 402 uses the toxicity evaluation model to extract threads that are responding appropriately to harmful comments from the SNS dump data input from the input unit 401, and uses such threads. Additional learning of the base interaction model is performed.

FIG. 9 shows a flowchart of a process for creating a dialogue model corresponding to various harmful comments in the dialogue system 100 according to the first embodiment, and FIG. 10 shows a data flow in this process.

The model management unit 402 first reads the base interaction model 1002, which is the basis of the harmful category corresponding interaction model, from an external device via the network 202 and stores it in the storage device 302 (step S901). The base dialogue model 1002 is a neural language model learned using a large amount of dialogue logs obtained from SNS etc., and utilizes a neural dialogue model published by a third party.

Note that the dialogue model to be used is a model that is trained using training data of the same language as the language handled by the dialogue processing of the dialogue system 100, or multilingual data containing a certain amount of the language, in an architecture handled by the model management unit 402. . If there is no trained dialogue model, the dialogue system 100 creates one by acquiring SNS dump data, shaping the data, and learning a language model.

Next, the model management unit 402 reads the formatted data 803 created by the input unit 401 in the process of creating a hazard evaluation model (step S902). Then, the model management unit 402 performs automatic annotation of toxicity on the formatted data using the toxicity evaluation model 403 created in the toxicity evaluation model creation process (step S903). Automatic annotation by the hazard evaluation model 403 is performed on a comment-by-comment basis, similar to the annotation in the hazard evaluation model creation process. Note that the automatic annotation here has a binary value of whether it is harmful or not.

Next, the model management unit 402 extracts data 1001 that includes a harmful comment and whose last reply comment is not harmful, using the harmfulness evaluation model 403 (step S904). Through this screening, dialogue data showing appropriate responses to harmful comments is collected.

Next, the model management unit 402 classifies the data 1001 extracted in step S904 into categories of toxicity using the category-specific toxicity evaluation model created in the hazard evaluation model creation process (step S905). Then, in the next step, the model management unit 402 performs additional learning of the interaction model corresponding to the harmful category.

First, the model management unit 402 focuses on one of the harmful categories (step S906). Then, the model management unit 402 performs additional learning on the base dialogue model 1002 read in step S901 using the data extracted in step S904, and creates a general-purpose hazard response dialogue model 1003 (step S907). The created general-purpose toxicity response interaction model 1003 is stored in the storage device 302.

Then, the model management unit 402 selects data of the category of interest from the data classified by category in step S905. The model management unit 402 further performs additional learning using the selected data on the general-purpose toxicity response interaction model 1003 that underwent additional learning in step S907, and creates a category response interaction model for the category.

If there is an unprocessed category (step S909: YES), the process returns to step S906 and an interaction model for the unprocessed category is created. In this way, interaction models for all categories are created (step S909: NO), and the process ends. In this way, performing additional learning using data from all categories before performing additional learning using categorical data has the effect of acquiring both generality for harmful utterances in general and specialization for specific harmful categories. .

<Interaction processing>
FIG. 11 shows a flowchart of dialogue processing in the dialogue system 100 according to the first embodiment. Dialogue processing uses the hazard evaluation model 403 created in the hazard evaluation model creation process and the multiple dialogue models for each harmful category created in the harm category compatible dialogue model creation process to determine the appropriate response to the harmful input. This is a process that outputs a response.

The interaction process is started when a user inputs text through the input device 303. First, the input unit 401 reads input data and outputs an input sentence included in the input data to the response generation unit 404 (step S1101). Here, the response generation unit 404 combines the dialogue history and the input sentence if the dialogue history management unit 407 holds the dialogue history. Next, the response generation unit 404 inputs the input sentence to each interaction model via the model management unit 402 (step S1102). Note that if the dialogue history management unit 407 does not hold the dialogue history, only the input sentence is input to each dialogue model. The model management unit 402 acquires response sentences for input sentences from all interaction models and outputs them to the response generation unit 404.

The response generation unit 404 inputs the response sentences received from the model management unit 402 to the response selection unit 405, and the response selection unit 405 evaluates the harmfulness of these response sentences (step S1103). The response selection unit 405 inputs each response sentence into the toxicity evaluation model 403 via the model management unit 402, and obtains a toxicity score for each response sentence. The harm score is the probability of the classification result calculated by the harm evaluation model 403 by passing the output through a softmax function, and takes a value between 0 and 1. The closer the harm score is to 1, the higher the probability that it is harmful.

Next, the response selection unit 405 selects the most appropriate one response sentence from the plurality of response sentences output by the response selection unit 405 based on the calculated harmfulness score. The response selection unit 405 first determines whether there is a response sentence whose toxicity score is less than a threshold value, that is, whether there is a safe response sentence (step S1104). If there is a response sentence with a toxicity score below the threshold (step S1104: YES), the response selection unit 405 selects the response sentence with the lowest toxicity score from among them (step S1105). Note that if there is only one response sentence that is less than the threshold, this process is omitted.

If there is no response sentence with a harmfulness score below the threshold (step S1104: NO), that is, if all the response sentences are determined to be harmful, the response selection unit 405 performs a preset response as an exception process. The fixed format response sentence is output (step S1106). The standard response sentence is, for example, a response indicating that the user does not agree with the harmful input sentence, such as "I don't agree with that statement." or "I don't understand." By processing in this manner, even if no appropriate response is obtained from any interaction model, harmful responses are prevented from being output.

Then, the response selection unit 405 outputs the selected response sentence to the output unit 406, and also outputs the input sentence and the selected response sentence to the conversation history management unit 407 as a conversation history (step S1107). The dialogue history management unit 407 adds the input sentence and response sentence output by the response selection unit 405 to the dialogue history. The dialogue history management unit 407 outputs the dialogue history to the response generation unit 404 when the next dialogue process is started.

According to the first embodiment described above, it is possible to provide a dialogue system that can return appropriate responses to various types of harmful comments.

According to the first embodiment, the dialogue models generated for each category generate appropriate responses to harmful comments classified into various categories, and the response selection unit selects the most appropriate response from among the responses generated by each dialogue model. You can choose the appropriate response. In other words, it is possible to return appropriate responses to various categories of harmful input sentences.

In this example, a hazard evaluation model for each hazard category is used for hazard evaluation, and a hazard score threshold is set for each hazard category, thereby making it possible to specify the importance of each hazard category. I will explain about it.

FIG. 12 is a block diagram showing an example of the functional configuration of the dialogue system according to this embodiment. The configuration is the same as that of the dialog system according to the first embodiment, except that the hazard evaluation model is a multiple model configuration as the hazard evaluation model group 1201. Moreover, the hazard evaluation model by category that constitutes the group of hazard evaluation models is the hazard evaluation model by category created in step S708 in the flowchart of the hazard evaluation model creation process described in FIG.

<Interaction processing>
FIG. 13 shows a flowchart of the interaction process according to the second embodiment. Steps S1301 to S1302 are the same as steps S1101 to S1102 in FIG. 11. In step S1303, the response generation unit 404 inputs the response sentences received from the model management unit 402 to the response selection unit 405, and the response selection unit 405 evaluates the harmfulness of these response sentences. The response selection unit 405 inputs each response sentence to the toxicity evaluation model group 1201 via the model management unit 402, and obtains a toxicity score from the category-based toxicity evaluation model for each response sentence.

FIG. 14 shows a harmfulness score table 1400 that is referred to by the response selection unit 405 in the dialog system, and is stored in various data 411 of the storage device 302. The harmfulness score table 1400 manages harmfulness scores of a plurality of response sentences. FIG. 14 shows examples of harmfulness scores for each category of three response sentences. For response sentences 1401 to 1403, scores for each category set 1405 are listed. 1406 is a total toxicity score calculated by the method described below.

The response selection unit 405 next determines whether there is a response sentence whose toxicity score is less than the threshold (step S1304). The response selection unit 405 determines whether all the toxicity scores obtained for each categorical hazard evaluation model are less than the threshold. That is, the response selection unit 405 determines the harmfulness of the response sentence by comparing the harmfulness score and threshold value of the harmfulness score table 1400 and the threshold value 1501 of the weighting coefficient management table 1500 for each attack target.

FIG. 15 shows a threshold value and weighting coefficient management table 1500, which is stored in various data 411 of the storage device 302. A threshold and weighting coefficient management table 1500 shows examples of thresholds 1501 for each category and weighting coefficients 1502 used for calculating a comprehensive toxicity score, which will be described later. In Figure 15, the threshold value for gender is 0.5, which is set lower than 0.6 for other categories, but this is a setting that emphasizes the harmfulness of gender and judges it as harmful even if the probability is low. It means that.

In the example of the response sentence shown in FIG. 14, the response sentence 1401 has a harmfulness score of 0.55 for both race/ethnicity and gender. In FIG. 15, the threshold value for race/ethnicity is 0.6, which is less than the threshold value, but the threshold value for gender is 0.5, so it is greater than the threshold value. Therefore, response selection unit 405 determines response sentence 1401 to be harmful. On the other hand, both response sentences 1402 and 1403 have toxicity scores below the threshold, and the response selection unit 405 determines these two response sentences as safe.

Since there are response sentences 1402 and 1403 whose toxicity score is less than the threshold (step S1304: YES), the response selection unit 405 calculates the overall toxicity score 1406 using the weighting coefficient 1502 for each response sentence. (Step S1305). The overall toxicity score is the maximum value obtained by multiplying the categorical toxicity scores by the respective specified weighting coefficients. When each harmfulness score for each category of the response sentence 1402 is multiplied by a weighting coefficient, the maximum gender for gender is 0.48, which is the product of the harmfulness score of 0.4 and the weighting coefficient of 1.2, and this value is the overall value of the response sentence 1402. This is the toxicity score. When calculated in a similar manner, the overall toxicity score of response sentence 1403 is 0.4.

Then, the response selection unit 405 selects the response with the lowest overall toxicity score (step S1306). In the example of FIG. 14, the response sentence 1403 has the lowest overall toxicity score of 0.4, so the response sentence 1403 is selected as the final output. Then, the response selection unit 405 outputs the selected response sentence to the output unit 406 and the dialogue history management unit 407 (step S1308).

According to the second embodiment described above, it is possible to provide an interaction system in which it is possible to arbitrarily set whether to treat all harmful categories equally or to change the importance of a specific category in determining the harmfulness.

In this example, by arbitrarily changing the configuration of a group of dialogue models, the corresponding harmful category can be changed, and the number of dialogue models to be used can be reduced, thereby making it possible to operate with limited resources. I will explain about it.

FIG. 16 is a block diagram showing an example of the functional configuration of the dialogue system according to this embodiment. In the third embodiment, the interaction model group 1600 is composed of a standard model group 1601, a mixed model group 1602, and a detailed model group 1603.

The standard model group 1601 has the same configuration as the dialog model group 408 of the dialog system according to the first embodiment, and is composed of the corresponding dialog models for each harmful category described in the first and second embodiments.

The mixed model group 1602 is a dialogue model that is a mixture of a plurality of harmful categories, and is, for example, a dialogue model configured to correspond to race/ethnicity and gender in one dialogue model.

The detailed model group 1603 is an interaction model created by subdividing the harmful categories of the standard model group 1601 into categories. For example, race/ethnicity can be subdivided into black, white, Native American, and gender can be subdivided into male, female, LGPT, etc. The configuration other than the dialog model group 1600 is the same as that of the dialog system according to the first embodiment.

In this embodiment, by properly using the standard model group 1601, mixed model group 1602, and detailed model group 1603, it is possible to deal with harmful categories that are particularly important and to save hardware resources for this purpose.

<Hazard assessment model creation process>
FIG. 17 shows a flowchart of the hazard evaluation model creation process by the model management unit 402 of the dialog system according to the third embodiment. Note that FIG. 17 shows only the changes to the hazard evaluation model creation process described in FIG. 7. The flowchart in FIG. 17 corresponds to steps S702 to S704 in FIG. 7, and the operations thereafter are the same as those in FIG.

The model management unit 402 focuses on one piece of data from the data formatted for learning the hazard evaluation model (step S1701).

Then, the model management unit 402 outputs the data to be annotated to the output device 304, and receives user input through the input device 303 (step S1702). At this time, both the main category and subcategory are registered. The main category is the category used in Example 1, and the subcategories are the aforementioned racial/ethnic categories such as black, white, and Native American. The main category and subcategories are defined in advance so that the user can select them during annotation.

If the registered data includes a plurality of main categories (step S1703: YES), the model management unit 402 registers the combination of categories as a co-occurring category (step S1704). The example of data 608 shown in FIG. 6 includes two categories, race/ethnicity and gender, and falls under this condition. Note that in the example of FIG. 6, two categories are assigned to one comment (comment 604), but in step S1703, it is determined whether or not a data unit, which is a series of comment strings, includes multiple categories. The condition is that

FIG. 18 shows a registration example of the co-occurrence category table 1800. The co-occurrence category table 1800 is stored in the various data 411 of the storage device 302. The co-occurrence category table 1800 manages the correspondence between co-occurrence category patterns 1801 and their frequencies 1802. Since race/ethnicity and gender in the example of FIG. 6 match the pattern 1803, the model management unit 402 adds the frequency of this pattern. Note that if the pattern has not been registered, the pattern is registered. The process moves to step S1705, and if the number of data for all categories is satisfied (step S1705: YES), the process moves to step S1706, and if it is not satisfied (step S1705: NO), the process returns to step S1701 and the next Annotate the data.

In step S1706, the model management unit 402 registers co-occurrence categories with a high frequency exceeding an arbitrary threshold as mixed categories. In the example of FIG. 18, the model management unit 402 registers co-occurrence categories 1803, 1804, and 1805 as a mixed category set, for example. Next, the model management unit 402 checks whether there are any categories that are not included in the mixed category set registered in step S1706 (step S1707).

All categories are classified as race/ethnicity, gender, religion, appearance, health (disability), politics/socioeconomics, and registered mixed categories are classified as "race/ethnicity, gender," "appearance, gender," and "race/ethnicity." ``Ethnicity, Politics/Socioeconomics'', ``Health (Disability)'' is not included in the mixed category set. If there is a category that is not included in the mixed category set (step S1707: YES), the model management unit 402 adds the category to the created mixed category set (step S1708). As a result, "race/ethnicity, gender," "appearance, gender," "race/ethnicity, politics/socioeconomics," and "health (disability)" become a mixed category set.

The subsequent processing is similar to step S705 and subsequent steps in FIG. 7, and the model management unit 402 creates a hazard evaluation model and a category-specific hazard evaluation model, and then ends the process. Each hazard assessment model is created for each main category, subcategory, and mixed category.

<Harmful category compatible dialogue model creation process>
The dialogue model creation process corresponding to harmful categories in the dialogue system according to the third embodiment is the same as the creation process itself except that the dialogue models to be created are main category, subcategory, and mixed category. Main category interaction models are standard model group 1601, subcategory interaction models are detailed model group 1603, and mixed category interaction models are mixed model group 1602.

<Interaction processing>
In the dialog processing of the dialog system according to the third embodiment, a dialog model to be used at startup is input by a designated user from the input unit 401. When a dialogue model is designated, the model management unit 402 executes the same dialogue processing as the dialogue system according to the first embodiment, if the designated dialogue model is the standard model group 1601. For example, if the user specifies the mixed model group 1602 as the interaction model, the quality of the response will be lower than when the standard model group 1601 is used, but the reduction in the number of interaction models used will improve the operation. Necessary hardware resources can be reduced.

Specifically, the time from receiving input from the user to outputting the response is reduced, and when operating in a client-server format, the number of clients that can be handled at once can be increased. can be expected. On the other hand, when using the detailed model group 1603, although the hardware resources required for processing increase, it is possible to generate a more appropriate response according to the subcategory.

Furthermore, the present invention is not necessarily limited to the standard model group 1601, the mixed model group 1602, and the detailed model group 1603, and it is also possible to combine interaction models from different model groups. For example, if it is necessary to configure a small number of models using the mixed model group 1602 due to hardware resource issues, but there is a need to output a more appropriate response for "discrimination against women," then the mixed model group 1602 and detailed model A model corresponding to the "female" category included in group 1603 may be used together.

According to the third embodiment described above, it is possible to provide a dialogue system in which the configuration of the dialogue model can be arbitrarily changed depending on the availability of hardware resources for operating the dialogue system or when there is a harmful category that should be particularly emphasized. can.

Note that the present invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above are described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace some of the configurations of each embodiment with other configurations.

Moreover, each of the above-mentioned configurations, functions, processing units, processing steps, etc. may be realized in part or in whole by hardware, for example, by designing an integrated circuit.

Further, the control lines and information lines are shown to be necessary for explanation purposes, and not all control lines and information lines are necessarily shown in the product. In reality, almost all components may be considered to be interconnected.

100 Dialogue system 101 Harmful category 1 compatible dialogue model 110 Input sentence 111 Response sentence 401 Input section 402 Model management section 403 Hazard evaluation model 404 Response generation section 405 Response selection section 406 Output section 407 Dialogue history management section 408 Dialogue model group

Claims (10)

An interaction system comprising an information processing device and a server connected to the information processing device via a network,
The server is
an input unit that receives input data from the information processing device;
For each attack target using input data, multiple interaction models are trained to output safe response sentences to harmful input data, and each generates a response sentence in response to input data,
A dialogue system comprising: a response selection unit that selects and outputs the most appropriate response sentence from a plurality of response sentences generated by the plurality of dialogue models based on a predetermined criterion. The dialogue system according to claim 1,
The response selection section includes:
Calculating the toxicity score of each of the plurality of response sentences output by the plurality of dialogue models using a toxicity evaluation model that determines whether the response sentence is harmful;
A dialogue system that selects a response sentence whose toxicity score is less than a predetermined threshold and the lowest. The dialogue system according to claim 1,
Each of the plurality of interaction models is
A dialogue system that learns from harmful input data and safe response sentences for each input data attack target. The dialogue system according to claim 3,
The server includes a model management unit that creates the plurality of interaction models,
The model management department includes:
Executing annotation on the input data input from the input unit using a hazard evaluation model,
Extracting input data that includes a safe response sentence to harmful input data from the input data input from the input section,
The extracted input data is classified according to the attack target, and
Using each classified input data, the base interaction model is additionally trained as a general-purpose hazard response interaction model,
A dialogue system that generates an attack target compatible dialogue model for each attack target by additionally learning the general-purpose harmfulness compatible dialogue model using input data classified into the attack target. The dialogue system according to claim 1,
The response selection section includes:
Calculating the harm score of the response sentences output by the plurality of dialogue models using a plurality of harm evaluation models that determine whether each attack target based on input data is harmful;
A dialogue system that selects a response sentence whose calculated harm score is less than a predetermined threshold and the lowest. The dialogue system according to claim 5,
The server is
a toxicity score table for managing toxicity scores for each attack target of the plurality of response sentences;
It has a storage device that stores thresholds and weighting coefficient management tables for managing harmfulness score thresholds and weighting coefficients for each attack target,
The response selection section includes:
Determine the harmfulness of the response sentence by comparing the harmfulness score of the harmfulness score table with the threshold value and the threshold value of the weighting coefficient management table for each attack target,
A dialogue system that selects the optimal response sentence from among safe response sentences based on the weighting coefficient for each attack target. The dialogue system according to claim 4,
The plurality of interaction models are:
A dialogue system that classifies attack targets based on input data into categories, and learns from harmful input data classified into a mixed category that is a mixture of a plurality of the categories and safe responses to the input data. The dialogue system according to claim 7,
The input section is
Specifying an attack target response interaction model to be used from among the interaction models included in the plurality of interaction models,
The model management unit is a dialogue system that generates a response sentence to input data using a designated attack target correspondence dialogue model. An interaction device having an input device that receives input data and an output device that outputs a response sentence,
A plurality of interaction models each of which is trained to output a safe response sentence to harmful input data for each attack target using the input data input by the input device, and each generates a response sentence in response to the input data. An interaction device comprising: a response selection unit that selects a response sentence from a plurality of response sentences based on a predetermined criterion and outputs the selected response sentence to the output device. A dialogue method using a dialogue system comprising an input part and an output part,
The plurality of dialogue models of the dialogue system are:
It has been trained to output safe responses to harmful input data,
Each of the plurality of interaction models generates a response sentence to the input data for each attack target based on the input data, and
A dialogue method that selects and outputs a response sentence based on a predetermined criterion from response sentences of each of the plurality of dialogue models.

Images