JP2021039537A - Response evaluation device, response evaluation method and computer program - Google Patents

Abstract

To appropriately extract elements of a dialogue useful for achieving the purpose of a response in a past dialogue.SOLUTION: A response evaluation device is provided with a usefulness calculation processing unit that calculates usefulness for achieving the purpose of a response based on a success or failure result of the purpose of the response for a dialogue text with respect to morphemes included in the dialogue text of a past response, and an evaluation processing unit that generates evaluation data of the morpheme relating to the achievement of the purpose of the response based on the usefulness.SELECTED DRAWING: Figure 1

Description

The present invention relates to a response evaluation device, a response evaluation method, and a computer program.

Conventionally, Patent Document 1 describes a topic model learning technique for clarifying a shared topic structure between speakers in dialogue. The topic model learning technique described in Patent Document 1 clarifies the topic structure such as "the B-th topic is dominant in the dialogue between Mr. E and Mr. F" in the dialogue, and "Mr. E spoke. A topic model that can capture information such as "The word of the Bth topic is C or D" is realized.

Japanese Unexamined Patent Publication No. 2015-45915

However, with the above-mentioned conventional techniques, it has not been possible to appropriately extract the elements of the dialogue useful for achieving the purpose of the response in the past dialogues.

The present invention has been made in consideration of such circumstances, and an object of the present invention is to appropriately extract elements of dialogue useful for achieving the purpose of reception in past dialogues of reception.

(1) One aspect of the present invention is a usefulness calculation for calculating the usefulness for achieving the purpose of the response based on the success or failure result of the purpose of the response for the dialogue text for the morphemes included in the dialogue text of the past response. It is a response evaluation device including a processing unit and an evaluation processing unit that generates evaluation data of the morpheme related to the achievement of the purpose of the response based on the usefulness.
(2) In one aspect of the present invention, the usefulness calculation processing unit has the success level of the purpose of the response in the dialogue text in which the usefulness calculation target morpheme appears, and the dialogue in which the usefulness calculation target morpheme does not appear. The response evaluation device according to (1) above, comprising a usefulness calculation unit for calculating the usefulness of the usefulness calculation target morpheme based on the success level of the purpose of the response in the text.
(3) In one aspect of the present invention, the evaluation processing unit includes an evaluation data generation unit that generates the evaluation data for a morpheme cluster in which the morpheme is clustered based on the characteristics of the morpheme (1). ) Or (2).
(4) In one aspect of the present invention, the evaluation data is based on the objective achievement rate for the response, the appearance rate in the dialogue text, and the success of the objective success for the response in the dialogue text for each morpheme cluster. The response evaluation device according to (3) above, which includes at least one of an appearance point and a failure appearance point in the dialogue text of the purpose failure for the response.
(5) In one aspect of the present invention, the evaluation processing unit generates a feature vector representing the characteristics of the one morpheme based on the combination with the other morpheme that appears together with the one morpheme. It is a response evaluation device according to any one of (3) or (4) above, which includes a characterizing unit.
(6) In one aspect of the present invention, the evaluation processing unit generates the morpheme cluster based on the distance between the feature vector of one morpheme and the feature vector of the other morpheme. The response evaluation device according to (5) above, which comprises a unit.
(7) In one aspect of the present invention, the evaluation processing unit calculates the usefulness representative value of the morpheme cluster based on the usefulness of the morpheme contained in the same morpheme cluster, and the usefulness representative value is The response evaluation device according to any one of (3) to (6) above, which includes a topic cluster extraction unit that extracts the morpheme cluster that is equal to or higher than a predetermined threshold as a topic cluster, and targets the topic cluster as a target for generating the evaluation data. Is.

(8) In one aspect of the present invention, the computer calculates the usefulness of the morphology included in the dialogue text of the past response for achieving the purpose of the response based on the success or failure result of the purpose of the response for the dialogue text. It is a computer program for executing a usefulness calculation processing step and an evaluation processing step for generating evaluation data of the form element related to the achievement of the purpose of the response based on the usefulness.

(9) In one aspect of the present invention, the response evaluation device determines the usefulness of the morpheme included in the dialogue text of the past response for achieving the purpose of the response based on the success or failure result of the purpose of the response for the dialogue text. It is a response evaluation method including a usefulness calculation processing step to be calculated and an evaluation processing step in which the response evaluation device generates evaluation data of the morpheme related to the achievement of the purpose of the response based on the usefulness.

According to the present invention, it is possible to appropriately extract the elements of the dialogue useful for achieving the purpose of the response in the dialogue of the past response.

It is a block diagram which shows the structural example of the response evaluation apparatus which concerns on one Embodiment. It is a figure which shows the structural example of the dialogue text database which concerns on one Embodiment. It is a figure which shows the structural example of the objective success or failure result database which concerns on one Embodiment. It is a figure which shows the structural example of the morpheme list which concerns on one Embodiment. It is a figure which shows the composition example of the dialogue text data with purpose success or failure which concerns on one Embodiment. It is a figure which shows the structural example of the all morpheme list with usefulness which concerns on one Embodiment. It is a figure which shows the structural example of the all morpheme feature quantity list which concerns on one Embodiment. It is a figure which shows the structure example of the all morpheme feature quantity list with the topic number which concerns on one Embodiment. It is a figure which shows the structure example of the all morpheme feature quantity list with topic number usefulness which concerns on one Embodiment. It is a figure which shows the structural example of the topic cluster morpheme feature list with usefulness which concerns on one Embodiment. It is a figure which shows the structural example of the evaluation data which concerns on one Embodiment. It is a figure which shows the structural example of the evaluation data which concerns on one Embodiment. It is a figure which shows the structural example of the evaluation data which concerns on one Embodiment. It is a flowchart which shows an example of the procedure of the response evaluation method which concerns on one Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration example of a response evaluation device according to an embodiment. In FIG. 1, the response evaluation device 1 includes a dialogue text database (dialogue text DB) 2, a purpose success / failure result database (purpose success / failure result DB) 3, a morpheme dictionary 4, a morpheme division unit 5, and a usefulness calculation processing unit. 10 and an evaluation processing unit 20 are provided.

Each function of the response evaluation device 1 is realized by the response evaluation device 1 having computer hardware such as a CPU (Central Processing Unit) and a memory, and the CPU executing a computer program stored in the memory. Will be done.
The response evaluation device 1 may be configured by using a general-purpose computer device, or may be configured as a dedicated hardware device. For example, the response evaluation device 1 may be configured by using a server computer connected to a communication network such as the Internet. Further, each function of the response evaluation device 1 may be realized by cloud computing.

The dialogue text DB2 is a database that stores dialogue texts of past responses. For example, a text (dialogue text) in which the dialogue between the respondent and the customer in the service center of the company is converted into a text is stored in the dialogue text DB2. Alternatively, for example, a dialogue text in which the dialogue between the respondent and the customer at the product sales store is converted into a text is stored in the dialogue text DB2. FIG. 2 is a diagram showing a configuration example of a dialogue text DB according to the present embodiment. As shown in FIG. 2, the dialogue text DB2 stores the dialogue identifier (dialogue ID) and the dialogue (dialogue text) in association with each other.
Here, the plurality of dialogue texts evaluated together have the same purpose of response. Therefore, as an example of the present embodiment, only the dialogue text of the same purpose is stored in one dialogue text DB2, and all the dialogue texts stored in the one dialogue text DB2 are evaluated together. set to target. Therefore, the dialogue texts for different purposes are stored in separate dialogue text DB2.
For example, the purpose of the response in "Case 1: Dialogue text of the response between the customer and the customer at the service center of the company" is that the customer is satisfied with the response, and "Case 2: The customer and the customer at the product sales store" It is assumed that the purpose of the response of "Dialogue text of the response" is that the customer purchases the product. In this case, since the purpose of the response is different between the case 1 and the case 2, the dialogue text DB 2 is provided separately in the case 1 and the case 2 so that the case 1 and the case 2 are evaluated separately.
As a modification of this embodiment, case 1 and case 2 are identifiablely stored in one dialogue text DB2, and when case 1 is to be evaluated, only case 1 is extracted, while case 2 is used. In the case of making the evaluation target, a target dialogue extraction unit for extracting only the case 2 may be provided.

The purpose success / failure result DB3 is a database that accumulates the success / failure results of the purpose of the response for each dialogue text stored in the dialogue text DB2. For example, the purpose of the service between the customer and the customer in the service center of the company is to satisfy the customer. For example, the purpose of dealing between the customer and the customer at the product sales store is that the customer purchases the product. FIG. 3 is a diagram showing a configuration example of the purpose success / failure result DB according to the present embodiment. As shown in FIG. 3, the purpose success / failure result DB3 stores the dialogue ID and the purpose success / failure “1: success” or “0: failure” of the dialogue text corresponding to the dialogue ID in association with each other.

The morpheme dictionary 4 is a dictionary for dividing a sentence into morphemes. The morpheme dividing unit 5 divides the dialogue text in the dialogue text DB 2 into morphemes using the morpheme dictionary 4, and generates a morpheme list for each dialogue text as illustrated in FIG. Also, a list of all morphemes (list of all morphemes) is generated.

The usefulness calculation processing unit 10 calculates the usefulness of the morpheme included in the dialogue text of the past response for achieving the purpose of the response based on the success / failure result of the purpose of the response for the dialogue text. The usefulness of a morpheme is the degree to which the morpheme is useful in achieving the purpose of the response in the dialogue.

The usefulness calculation processing unit 10 includes a purpose success / failure result coupling unit 11 and a usefulness calculation unit 12.

The purpose success / failure result combining unit 11 associates the dialogue text in the dialogue text DB2 with the purpose success / failure "1: success" or "0: failure" in the purpose success / failure result DB3 using the dialogue ID as a key. As a result, as illustrated in FIG. 5, dialogue text data with a purpose success / failure is generated in which the dialogue text having the same dialogue ID and the purpose success / failure “1: success” or “0: failure” are associated with each other.

The usefulness calculation unit 12 calculates the usefulness of each morpheme for the dialogue text stored in the dialogue text DB2 by using the dialogue text data with the purpose success or failure and the morpheme list. The usefulness calculation unit 12 determines the success level of the purpose of the response in the dialogue text in which the morpheme whose usefulness is calculated (the morpheme whose usefulness is calculated) appears, and the response in the dialogue text in which the usefulness calculation target morpheme does not appear. The usefulness of the usefulness calculation target morpheme is calculated based on the success degree of the purpose of. An example of the usefulness calculation method will be described below.

[Example of usefulness calculation method]
The usefulness calculation unit 12 calculates the usefulness by the following formula.
Usefulness = (A ÷ (A + B)) ÷ (C ÷ (C + D))
However, A is the total number of dialogue texts for which the purpose is "1: success" among the dialogue texts in which the usefulness calculation target morpheme appears. B is the total number of dialogue texts for which the purpose is "0: failure" among the dialogue texts in which the usefulness calculation target morpheme appears. C is the total number of dialogue texts whose purpose is success or failure "1: success" among the dialogue texts in which the usefulness calculation target morpheme does not appear. D is the total number of dialogue texts whose purpose is success or failure "0: failure" among the dialogue texts in which the usefulness calculation target morpheme does not appear.
The above is an explanation of an example of the usefulness calculation method.

The usefulness calculation unit 12 adds the usefulness of each morpheme to the total morpheme list, and generates a total morpheme list with usefulness as illustrated in FIG.

The evaluation processing unit 20 generates evaluation data of morphemes related to the achievement of the purpose of the response based on the usefulness. The evaluation processing unit 20 includes a morpheme characterization unit 21, a morpheme clustering unit 22, a usefulness coupling unit 23, a topic cluster extraction unit 24, and an evaluation data generation unit 25.

The morpheme characterization unit 21 calculates the feature amount of each morpheme included in the total morpheme list. As an example of the present embodiment, the morpheme characterization unit 21 generates a feature vector representing the characteristics of the one morpheme based on the combination with another morpheme that appears together with the one morpheme.

For example, the morpheme characterization unit 21 sets a vector (feature amount vector) of each morpheme representing the relationship between the morphemes based on the combination of one morpheme in the arrangement of morphemes in the sentence and the morphemes before and after the morpheme. Generate. Machine learning may be used for the relationships between morphemes. For example, by deep learning the combination of one morpheme in the arrangement of morphemes in a sentence and the morphemes before and after the morpheme, a feature vector of each morpheme representing the relationship between the morphemes may be generated.

The morpheme characterization unit 21 generates a list of all morpheme features associated with morphemes and features. FIG. 7 shows a configuration example of a list of all morpheme features. The list of all morpheme features illustrated in FIG. 7 is a list in which morphemes and feature vectors are described in association with each other.

The morpheme clustering unit 22 clusters each morpheme into an arbitrary number of clusters (morpheme clusters) based on the feature amount of each morpheme in the total morpheme feature amount list. A topic number is assigned to each morpheme cluster. As an example of this embodiment, the morpheme clustering unit 22 generates a morpheme cluster based on the distance between the feature vector of one morpheme and the feature vector of another morpheme. The shorter the distance between the feature vectors, the higher the similarity between the corresponding morphemes. For example, a hierarchical clustering method may be used. In the hierarchical clustering method, morpheme clusters are determined in order from the combination of the most similar morphemes, and when all the morphemes are finally combined into one morpheme cluster, clustering is terminated. The number of clusters may be set in advance. For example, the upper limit of the number of clusters is set in advance, and clustering may be terminated when the upper limit of the number of clusters is reached.

The morpheme clustering unit 22 reflects the topic number of the clustering result in the list of all morpheme features. FIG. 8 shows a configuration example of a list of all morpheme features with topic numbers. As illustrated in FIG. 8, in the list of all morpheme features with topic numbers, the topic numbers of the morpheme clusters to which the morphemes belong are described in association with each morpheme.

The usefulness combining unit 23 combines the list of all morphemes with usefulness (see FIG. 6) and the list of all morphemes with topic numbers (see FIG. 8), and all with topic numbers illustrated in FIG. Generate a morpheme feature list. In FIG. 9, in the list of all morpheme feature quantities with topic number usefulness, the morpheme, the feature quantity vector, the topic number, and the usefulness are described in association with each morpheme.

The topic cluster extraction unit 24 calculates the usefulness representative value of the morpheme cluster based on the usefulness of the morphemes included in the same morpheme cluster, and extracts the morpheme cluster whose usefulness representative value is equal to or higher than a predetermined threshold value as a topic cluster. To do. Specifically, in the list of all morpheme features with topic number usefulness, the usefulness representative value of the topic number is calculated based on the usefulness of the morpheme of the same topic number. The representative value of usefulness is, for example, an average value. Next, the morpheme cluster of the topic number whose usefulness representative value is equal to or higher than a predetermined threshold value is determined as the topic cluster.
As the representative value of usefulness, for example, an average value, a median value, a minimum value, or the like can be used.

The topic cluster extraction unit 24 generates a topic cluster morpheme feature list with usefulness by extracting only topic clusters from the list of all morpheme features with topic number usefulness. FIG. 10 shows a configuration example of a topic cluster morpheme feature list with usefulness. In the topic cluster morpheme feature quantity list with usefulness exemplified in FIG. 10, only the morphemes belonging to the topic cluster are described in association with the feature quantity vector, the topic number, and the usefulness.

The evaluation data generation unit 25 generates evaluation data for the morpheme cluster. As the evaluation data, for each morpheme cluster, there are the purpose achievement rate for the response, the appearance rate in the dialogue text, the success appearance point in the dialogue text for the purpose success for the response, the failure appearance point in the dialogue text for the purpose failure for the response, and the like. .. Evaluation data containing at least one of these is generated.

As an example of this embodiment, the evaluation data generation unit 25 generates evaluation data only for the topic cluster. The reason for this will be explained. Among the plurality of morpheme clusters generated by the morpheme clustering unit 22, there is a possibility that those having relatively high usefulness to those having relatively low usefulness are mixed. Therefore, by generating the evaluation data only for the topic clusters having higher usefulness, more accurate evaluation data can be efficiently generated and provided.

As illustrated in FIG. 11, the evaluation data generation unit 25 calculates the number of appearances, the appearance rate, the target achievement rate, and the appearance points for each topic cluster.

The number of occurrences is the number of dialogue texts that belong to the topic cluster to be calculated and include a predetermined number of morphemes. An arbitrary number is set in advance as the predetermined number. The evaluation data generation unit 25 calculates the number of occurrences of the topic cluster to be calculated based on the topic cluster morpheme feature list with usefulness and the morpheme list of each dialogue text.

The appearance rate is the ratio of the number of appearances for the topic cluster to be calculated to the total number of dialogue texts in the dialogue text DB2. The formula for calculating the appearance rate (percentage) is shown in the following formula.
Appearance rate = "(Number of appearances for the topic cluster to be calculated) / (Total number of dialogue texts in dialogue text DB2)" x 100 [%]

The purpose achievement rate is the ratio of the dialogue text Tb whose purpose is success or failure "1: success" among the dialogue text Ta counted in the number of appearances of the topic cluster to be calculated. The following formula shows the calculation formula of the purpose achievement rate (percentage).
Objective achievement rate = "(number of dialogue texts Tb) ÷ (number of dialogue texts Ta)" x 100 [%]

The appearance point is a part in which the morphemes belonging to the calculation target topic cluster are relatively large in the arrangement of the morphemes in the dialogue text Ta counted in the number of appearances of the calculation target topic cluster. As an example of this embodiment, the dialogue text is divided into three equal parts, the first half part, the middle part part, and the second half part according to the time series of the dialogue, and belongs to the calculation target topic cluster among the first half part, the middle part part, and the second half part. A portion containing a relatively large amount of morpheme is determined as an appearance point. However, if the first half, the middle part, and the second half are the same, the appearance point is defined as "whole".

More specifically, in the evaluation data generation unit 25, the number of morphemes belonging to the topic cluster to be calculated and any number of morphemes are included in the first half and the number of dialogue texts in the middle part. And the number of dialogue texts included in the latter half. Next, the evaluation data generation unit 25 compares the number of dialogue texts for each part of the calculation result, and based on the comparison result, the appearance points “first half (first half part)”, “middle stage (middle part)”, Judge "second half (second half)" or "whole".

Further, as an example of the present embodiment, as the appearance points, among the dialogue text Ta, the appearance point (success appearance point) in the dialogue text Tb whose purpose is success or failure "1: success" and the purpose success or failure "0: failure" are used. The appearance point (failure appearance point) in a certain dialogue text Tc is obtained.

The evaluation processing unit 20 outputs the evaluation data generated by the evaluation data generation unit 25. As an evaluation data output method, for example, display of evaluation data on a display screen of a display device such as a liquid crystal display device, print output of evaluation data, transmission of evaluation data to a predetermined destination by communication data such as e-mail, etc. Can be mentioned.

12 and 13 are diagrams showing an example of screen display of evaluation data. In FIG. 12, the purpose achievement rate and the appearance rate for each topic number are displayed on the screen. In FIG. 13, success appearance points (object achievement column) and failure appearance points (object failure column) for each topic number are displayed on the screen.

According to the evaluation data according to the present embodiment, the evaluation contents such as the objective achievement rate, the appearance rate, the success appearance point, and the failure appearance point are shown for each morpheme cluster. Therefore, the response is based on the evaluation contents of each morpheme cluster. The morpheme cluster to be reflected in the dialogue in is appropriately selected. For example, using a morpheme belonging to a morpheme cluster with a relatively good goal achievement rate, a model response example is created in which a morpheme belonging to the morpheme cluster appears with reference to the success appearance point and the failure appearance point of the morpheme cluster. You may. By using the model response example, a person with low response skill or an inexperienced response can contribute to the improvement of achievement of the purpose of response.

Further, by generating and outputting the evaluation data only for the topic cluster, more accurate evaluation data can be efficiently generated and provided.

Next, with reference to FIG. 14, the response evaluation method according to the present embodiment will be described. FIG. 14 is a flowchart showing an example of the procedure of the response evaluation method according to the present embodiment.

In advance, the dialogue text in the dialogue text DB2 is divided into morphemes, and a morpheme list is generated for each dialogue text as illustrated in FIG. In addition, a list of all morphemes (list of all morphemes) is generated in advance.

(Step S1) The purpose success / failure result combining unit 11 associates the dialogue text in the dialogue text DB2 with the purpose success / failure “1: success” or “0: failure” in the purpose success / failure result DB3 using the dialogue ID as a key. As a result, as illustrated in FIG. 5, dialogue text data with a purpose success / failure is generated in which the dialogue text having the same dialogue ID and the purpose success / failure “1: success” or “0: failure” are associated with each other.

(Step S2) The usefulness calculation unit 12 calculates the usefulness of each morpheme for the dialogue text stored in the dialogue text DB2 by using the dialogue text data with the purpose success or failure and the morpheme list. Specifically, the usefulness calculation unit 12 determines the success level of the purpose of the response in the dialogue text in which the usefulness calculation target morpheme appears and the success level of the response purpose in the dialogue text in which the usefulness calculation target morpheme does not appear. Based on the above, the usefulness of the usefulness calculation target morpheme is calculated. The usefulness calculation unit 12 adds the usefulness of each morpheme to the total morpheme list, and generates a total morpheme list with usefulness as illustrated in FIG.

(Step S3) The morpheme characterization unit 21 calculates the feature amount of each morpheme included in the total morpheme list. Specifically, the morpheme characterization unit 21 generates a feature vector representing the characteristics of the one morpheme based on the combination with other morphemes that appear together with the one morpheme. As illustrated in FIG. 7, the morpheme characterization unit 21 generates a list of all morpheme features in which morphemes and features are associated with each other.

(Step S4) The morpheme clustering unit 22 clusters each morpheme into an arbitrary number of morpheme clusters based on the feature amount of each morpheme in the total morpheme feature amount list. Specifically, the morpheme clustering unit 22 generates a morpheme cluster based on the distance between the feature vector of one morpheme and the feature vector of another morpheme. As illustrated in FIG. 8, the morpheme clustering unit 22 reflects the topic number of the clustering result in the list of all morpheme features.

(Step S5) The usefulness combining unit 23 combines the list of all morphemes with usefulness (see FIG. 6) and the list of all morphemes with topic numbers (see FIG. 8), and the topic number exemplified in FIG. Generate a list of all morpheme features with usefulness.

(Step S6) The topic cluster extraction unit 24 calculates the usefulness representative value of the morpheme cluster based on the usefulness of the morphemes included in the same morpheme cluster, and selects the morpheme cluster whose usefulness representative value is equal to or higher than a predetermined threshold value. Extract as a topic cluster. Specifically, in the list of all morpheme features with topic number usefulness, the usefulness representative value of the topic number is calculated based on the usefulness of the morpheme of the same topic number. Next, the morpheme cluster of the topic number whose usefulness representative value is equal to or higher than a predetermined threshold value is determined as the topic cluster. As illustrated in FIG. 10, the topic cluster extraction unit 24 generates a topic cluster morpheme feature list with usefulness by extracting only topic clusters from the list of all morpheme features with topic number usefulness.

(Step S7) The evaluation data generation unit 25 determines the goal achievement rate for the response, the appearance rate in the dialogue text, the success appearance point in the dialogue text for the purpose success for the response, and the dialogue text for the purpose failure for the response for each morpheme cluster. Generates evaluation data including failure occurrence points in. The evaluation data generation unit 25 may generate evaluation data only for the topic cluster.

(Step S8) The evaluation processing unit 20 outputs the evaluation data generated by the evaluation data generation unit 25.

According to the above-described embodiment, the response evaluation device 1 calculates the usefulness of the morpheme included in the dialogue text of the past response for achieving the purpose of the response based on the success / failure result of the purpose of the response for the dialogue text. A usefulness calculation processing unit 10 is provided, and an evaluation processing unit 20 that generates evaluation data of the morpheme related to the achievement of the purpose of the response based on the usefulness. As a result, it is possible to appropriately extract the elements of the dialogue that are useful for achieving the purpose of the response in the dialogue of the past response.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like within a range not deviating from the gist of the present invention are also included.

Further, a computer program for realizing the functions of the above-described devices may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The "computer system" referred to here may include hardware such as an OS and peripheral devices.
The "computer-readable recording medium" is a writable non-volatile memory such as a flexible disk, a magneto-optical disk, a ROM, or a flash memory, a portable medium such as a DVD (Digital Versatile Disc), or a built-in computer system. A storage device such as a hard disk.

Furthermore, the "computer-readable recording medium" is a volatile memory inside a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line (for example, DRAM (Dynamic)). It also includes those that hold the program for a certain period of time, such as Random Access Memory)).
Further, the program may be transmitted from a computer system in which this program is stored in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the above-mentioned functions. Further, a so-called difference file (difference program) may be used, which can realize the above-mentioned functions in combination with a program already recorded in the computer system.

1 ... Response evaluation device, 2 ... Dialogue text database (dialogue text DB), 3 ... Purpose success / failure result database (purpose success / failure result DB), 4 ... Morpheme dictionary, 5 ... Morpheme division unit, 10 ... Usefulness calculation processing unit, 11 ... Purpose success / failure result combination unit, 12 ... Usefulness calculation unit, 20 ... Evaluation processing unit, 21 ... Morpheme characterization unit, 22 ... Morpheme clustering unit, 23 ... Usefulness combination unit, 24 ... Topic cluster extraction unit, 25 ... Evaluation Data generator

Claims (9)

For the morphemes included in the dialogue text of the past response, the usefulness calculation processing unit that calculates the usefulness for achieving the purpose of the response based on the success / failure result of the purpose of the response for the dialogue text, and
An evaluation processing unit that generates evaluation data of the morpheme related to the achievement of the purpose of the response based on the usefulness.
A response evaluation device equipped with. The usefulness calculation processing unit
Usefulness calculation The usefulness is calculated based on the success level of the purpose of the response in the dialogue text in which the target morpheme appears and the success level of the response purpose in the dialogue text in which the usefulness calculation target morpheme does not appear. A usefulness calculation unit for calculating the usefulness of the target morpheme is provided.
The response evaluation device according to claim 1. The evaluation processing unit
It is provided with an evaluation data generation unit that generates the evaluation data for a morpheme cluster in which the morphemes are clustered based on the characteristics of the morphemes.
The response evaluation device according to any one of claims 1 or 2. The evaluation data includes the objective achievement rate for the response, the appearance rate in the dialogue text, the success appearance point in the dialogue text for the objective success for the response, and the objective failure for the response for each morpheme cluster. Includes at least one of the failure occurrence points in the dialogue text of
The response evaluation device according to claim 3. The evaluation processing unit
A morpheme characterization unit that generates a feature vector representing the characteristics of the one morpheme based on a combination with the other morpheme that appears together with the one morpheme is provided.
The response evaluation device according to any one of claims 3 or 4. The evaluation processing unit
A morpheme clustering unit that generates a morpheme cluster based on the distance between the feature vector of one morpheme and the feature vector of another morpheme is provided.
The response evaluation device according to claim 5. The evaluation processing unit
A topic that calculates the usefulness representative value of the morpheme cluster based on the usefulness of the morpheme included in the same morpheme cluster, and extracts the morpheme cluster whose usefulness representative value is equal to or higher than a predetermined threshold value as a topic cluster. Equipped with a cluster extractor
Targeting the topic cluster as the evaluation data generation target,
The response evaluation device according to any one of claims 3 to 6. On the computer
For the morphemes included in the dialogue text of the past response, the usefulness calculation processing step for calculating the usefulness for achieving the purpose of the response based on the success / failure result of the purpose of the response for the dialogue text, and
An evaluation processing step for generating evaluation data of the morpheme regarding the achievement of the purpose of the response based on the usefulness, and an evaluation processing step.
A computer program to run. A usefulness calculation processing step in which the response evaluation device calculates the usefulness for achieving the purpose of the response based on the success or failure result of the purpose of the response for the dialogue text for the morphemes included in the dialogue text of the past response.
An evaluation processing step in which the response evaluation device generates evaluation data of the morpheme relating to the achievement of the purpose of the response based on the usefulness.
Response evaluation method including.

Images