JP2020004137A - Evaluation device, evaluation method, and evaluation program - Google Patents

Abstract

To appropriately evaluate a response of an operator including difficulty of a case.SOLUTION: An evaluation device 10 acquires data transmitted and received by a terminal of an operator who receives an inquiry from a user, extracts a predetermined keyword from the acquired data, and classifies the inquiry for every content based on the keyword. Then, the evaluation device 10 gives a score showing difficulty of the inquiry according to the classified content, and outputs an evaluation result of the operator based on the given score.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an evaluation device, an evaluation method, and an evaluation program.

  In recent years, shortage of workers has been a problem due to a high turnover rate in contact centers. The cause is that there are many items to be remembered by the operator, and even though the job is stressful in complaint processing or the like, the results are not properly evaluated. 2. Description of the Related Art Conventionally, as a KPI (Key Performance Indicator) of a contact center, it is known that an operator is evaluated using quantitative data such as a response rate and a time until a response.

JP-A-2017-117161

  However, conventionally, since the operator is evaluated using quantitative data, there is a problem that the operator cannot be appropriately evaluated. For example, quantitative data such as the response rate and the time until a response require a long processing time when processing a case that requires advanced skills, such as a highly difficult inquiry or complaint, or a case that stresses the operator. In many cases, the evaluation may be lower than in the case of processing a case with low difficulty. As described above, there is a case where the operator cannot be appropriately evaluated, so that the motivation of the operator cannot be maintained and the turnover rate cannot be reduced.

  In order to solve the above-described problems and achieve the object, an evaluation device of the present invention includes an acquisition unit that acquires data transmitted and received by a terminal device of an operator that responds to an inquiry from a user, and an acquisition unit that acquires the data. A predetermined keyword is extracted from the data, and a classifying unit that classifies the inquiry for each content based on the keyword, and a score indicating the degree of difficulty of the inquiry is given according to the content classified by the classifying unit. It is characterized by having an assigning unit and an output unit for outputting the evaluation result of the operator based on the score assigned by the assigning unit.

  Further, the evaluation method of the present invention is an evaluation method executed by the evaluation device, wherein the obtaining step obtains data transmitted and received by the terminal device of the operator responding to the inquiry from the user, and the obtaining method obtains the data by the obtaining step. A predetermined keyword is extracted from the data, a classification step of classifying the inquiry for each content based on the keyword, and a score indicating the difficulty level of the inquiry is given according to the content classified by the classification step. And an output step of outputting an evaluation result of the operator based on the score given by the assigning step.

  Further, the evaluation program of the present invention is an acquisition step of acquiring data transmitted and received by a terminal device of an operator in response to an inquiry from a user, and extracts a predetermined keyword from the data acquired by the acquisition step, and A classifying step of classifying the inquiry for each content based on the content, an assigning step of assigning a score indicating the degree of difficulty of the inquiry according to the content classified by the sorting step, and a score assigned by the assigning step And outputting an evaluation result of the operator based on the program.

  ADVANTAGE OF THE INVENTION According to this invention, there exists an effect that an operator can be evaluated appropriately.

FIG. 1 is a block diagram illustrating a configuration example of the evaluation system according to the first embodiment. FIG. 2 is a block diagram illustrating a configuration example of the evaluation device according to the first embodiment. FIG. 3 is a diagram illustrating an example of data stored in the score setting table storage unit. FIG. 4 is a diagram illustrating an example of data stored in the score setting table storage unit. FIG. 5 is a diagram illustrating an example of data stored in the score setting table storage unit. FIG. 6 is a diagram illustrating an example of data stored in the evaluation management table storage unit. FIG. 7 is a diagram illustrating a display example of the evaluation result of the operator in the current case. FIG. 8 is a diagram showing a display example of the evaluation result of the operator in the last month. FIG. 9 is a diagram illustrating a display example of an evaluation result of an operator for each content of an inquiry. FIG. 10 is a diagram illustrating a display example of a relative evaluation result in a group of operators. FIG. 11 is a flowchart illustrating an example of the flow of a score assignment process in the evaluation device according to the first embodiment. FIG. 12 is a flowchart illustrating an example of the flow of an evaluation result output process in the evaluation device according to the first embodiment. FIG. 13 is a diagram illustrating a computer that executes a program.

  Hereinafter, embodiments of an evaluation device, an evaluation method, and an evaluation program according to the present application will be described in detail with reference to the drawings. The embodiment is not intended to limit the evaluation device, the evaluation method, and the evaluation program according to the present application.

[First Embodiment]
In the following embodiments, the configuration of the evaluation system, the configuration of the evaluation device 10, and the flow of processing of the evaluation device 10 according to the first embodiment will be described in order, and finally, the effects of the first embodiment will be described.

[Configuration of evaluation system]
FIG. 1 is a block diagram illustrating a configuration example of the evaluation system according to the first embodiment. The evaluation system according to the first embodiment includes an evaluation device 10 and a plurality of terminal devices 20A to 20F. The plurality of terminal devices 20A to 20C are connected to a company NW (Network) 40A, and the plurality of terminal devices 20D to 20F are connected to a company NW 40B.

  The evaluation device 10 is connected to a plurality of terminal devices 20A to 20C via the Internet 30 and the company NW 40A. Further, the evaluation device 10 is connected to a plurality of terminal devices 20D to 20F via the Internet 30 and the company NW 40B. The company NW 40A and the company NW 40B are networks of different companies. Although the example of FIG. 1 illustrates an example in which the evaluation device 10 is connected to a plurality of companies NWs 40A and 40B, the evaluation device 10 may be connected to only one company NW 40. The configuration illustrated in FIG. 1 is merely an example, and the specific configuration and the number of devices are not particularly limited. Further, when the terminal devices 20A to 20F and the company NWs 40A and 40B are described without particular distinction, they are referred to as the terminal device 20 and the company NW 40.

  The evaluation device 10 acquires voice data and text data transmitted and received by the terminal devices 20A to 20F of the operator responding to the inquiry from the user. Then, the evaluation device 10 extracts a predetermined keyword from the acquired voice data or text data, classifies the inquiry according to the content based on the keyword, and according to the classified content, a score indicating the difficulty level of the inquiry. Is given. Thereafter, the evaluation device 10 outputs the evaluation results of the operator based on the assigned scores to the terminal devices 20A to 20F.

  The terminal devices 20A to 20F are devices used by an operator such as an IP telephone and a desktop PC. For example, the terminal device 20 receives an inquiry by voice from a user's telephone or an inquiry by communication means using text such as chat or mail.

[Configuration of evaluation device]
Next, the configuration of the evaluation device 10 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a configuration example of the evaluation device according to the first embodiment. As shown in FIG. 2, the evaluation device 10 includes a communication processing unit 11, a control unit 12, and a storage unit 13. The processing of each unit of the evaluation device 10 will be described below. Note that each function of the evaluation device 10 may be distributed to a plurality of devices.

  The communication processing unit 11 controls communication regarding various types of information. For example, the communication processing unit 11 transmits the evaluation result of the operator to the terminal device 20.

  The storage unit 13 stores data and programs necessary for various processes performed by the control unit 12, and particularly includes a score setting table storage unit 13a and an evaluation management table storage unit 13b closely related to the present invention. For example, the storage unit 13 is a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The data setting method specified in each table described below may be set manually or may be set automatically using a program or the like.

  The score setting table storage unit 13a stores a score setting table in which inquiries are associated with scores indicating difficulty levels corresponding to the inquiries. For example, as illustrated in FIG. 3, the score setting table storage unit 13a stores “inquiry classification”, “evaluation classification”, “difficulty basic score”, “number of inquiries within a specified period”, “ A table associated with the score is stored.

  Here, in the “inquiry classification”, a combination of keywords used to classify the content of the inquiry is set. The “evaluation classification” is an item defined assuming evaluation items such as operator skill and load, and is set as an evaluation classification in association with the “inquiry classification”. In the “basic difficulty score”, a score indicating the difficulty level of the inquiry is set corresponding to the classification for evaluation.

  In the “number of inquiries within specified period”, the number of inquiries of each inquiry content within the specified period is stored. “Score” is a score actually given at the time of evaluation. The “score” is a value obtained by correcting the “basic level of difficulty” by taking into account the “number of inquiries within a specified period”. In other words, it is considered that an inquiry with a high inquiry frequency is easily handled because the operator is accustomed to it. On the other hand, it is assumed that it is difficult to handle cases that are not described in an operator's manual or the like and cases that have a low inquiry frequency. Therefore, the score setting table storage unit 13a stores the counted number of inquiries as a parameter for correcting “score”. The update of the “score” may be performed in real time or batch.

  In addition, the score setting table storage unit 13a stores a table that stores scores to be added or subtracted in association with the user's emotion when making an inquiry. For example, as illustrated in FIG. 4, the score setting table storage unit 13a stores “evaluation classification”, which is an evaluation item according to the user's emotion or change in emotion, and “score” given at the time of evaluation. Is stored in the table. In the example of FIG. 4, the score setting table storage unit 13a stores, for a case in which the user is angry (evaluation classification “emotion (anger)”), a score given as a result of a case with a high degree of difficulty. When the emotion is changed from a state where the user is angry to a state where the user is happy, the score given to the user is considered to have been good and the user's anger could be calmed down. 5 ".

  The score setting table storage unit 13a stores a table that stores scores to be added or subtracted in association with inquiries requiring special skills and how to respond to inquiries. For example, as illustrated in FIG. 5, the score setting table storage unit 13a stores “evaluation classification”, which is an evaluation item according to an inquiry requiring a special skill or a manner of responding to the inquiry, at the time of evaluation. A table in which the “score” is associated is stored.

  The evaluation management table storage unit 13b stores a table for managing the evaluation of the degree of difficulty for each inquiry. For example, as illustrated in FIG. 6, the evaluation management table storage unit 13b stores “date and time” indicating the date and time when the inquiry was made, “inquiry No.” that uniquely identifies the inquiry, and an operator who responded to the inquiry. And a table in which "operator ID" for uniquely identifying "" and "difficulty of inquiry" are stored. The “difficulty of inquiry” includes “inquiry classification”, “evaluation classification”, “score”, and “additional flag” indicating an item which is added or deducted according to the response of the operator.

  The control unit 12 has an internal memory for storing programs and required data defining various processing procedures and the like, and executes various processing by using these. Particularly, those closely related to the present invention include: It has an acquisition unit 12a, a classification unit 12b, an assignment unit 12c, and an output unit 12d. Here, the control unit 12 is an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

  The acquisition unit 12a acquires data transmitted and received by the terminal device 20 of the operator in response to an inquiry from the user. For example, when the user and the operator are communicating via a telephone call, the acquiring unit 12a acquires voice data including the conversation content between the user and the operator, or text data in which the conversation is converted into text. When acquiring the voice data, the acquiring unit 12a separates the conversation between the user and the operator and converts the conversation into a text using a known speech recognition technology. Note that the speech recognition technology is not particularly limited, but accuracy can be improved by using a speech recognition technology using AI (Artificial Intelligence).

  In addition, for example, when the user and the operator are communicating with each other using text data such as chat and mail, the obtaining unit 12a obtains the text data. Further, the acquisition unit 12a may acquire the data at any timing, may acquire the data in real time, or may start the call by using the fact that the call ends after the start of the call. The data may be acquired when a predetermined condition is satisfied, such as acquiring data for a certain period from to the end.

  The classification unit 12b extracts a predetermined keyword from the data acquired by the acquisition unit 12a, and classifies the inquiry for each content based on the keyword. For example, the classifying unit 12b performs a morphological analysis process and a keyword extraction process using a known text mining technique, and extracts a keyword included in the textualized conversation data. Although the text mining technique is not particularly limited, accuracy can be improved by using a text mining technique using AI. Then, the classification unit 12b refers to a table (for example, see FIG. 3) of the score setting table storage unit 13a, searches for an “inquiry class” that matches the extracted keyword combination, and sets the “inquiry class” that matches. Identify the corresponding “evaluation category”. When a plurality of “inquiry classifications” correspond to one inquiry, the evaluation classification corresponding to the corresponding “inquiry classification” is specified.

  Explaining with a specific example, for example, when the keywords “net” and “application” are included as keywords, the classification unit 12b refers to the table of the score setting table storage unit 13a (see FIG. 3). Specify “Inquiry about services / Internet” as the classification for evaluation. Further, for example, when the keywords “net” and “cancellation” are included as keywords, the classification unit 12b also specifies “inquiry about service / internet” as the evaluation classification.

  Further, for example, when the keywords “net” and “not connected” are included as keywords, the classification unit 12b refers to the table of the score setting table storage unit 13a and sets “technical inquiry / internet connection” as the evaluation classification. ".

  Note that the keywords extracted by the classifying unit 12b can also be used for trend analysis of the contents of inquiries to the center. Further, among the keywords extracted by the classifying unit 12b, a keyword that has abnormally increased during a certain period can be quickly detected, and if a problem occurs, the company can respond promptly.

  The assigning unit 12c assigns a score indicating the difficulty level of the inquiry according to the content classified by the classifying unit 12b. For example, the assigning unit 12c refers to the table of the score setting table storage unit 13a to assign a score corresponding to the evaluation classification specified by the classification unit 12b. Explaining by giving a specific example, when the evaluation classification specified by the classification unit 12b is “inquiry about service / Internet”, the giving unit 12c stores the table in the score setting table storage unit 13a (see FIG. 3). , It is determined that the corresponding score is “1”, and the score “1” is stored in the evaluation management table storage unit 13b.

  Further, the assigning unit 12c adds or subtracts a score according to the emotion of the user or a change in the emotion determined from the voice data or the text data. For example, the providing unit 12c determines the user's emotion using a known emotion recognition technology, and searches whether data matching the determined emotion is stored in the table (FIG. 4) of the score setting table storage unit 13a. I do. As a result, when searching for data matching the determined emotion, the assigning unit 12c assigns a corresponding score. More specifically, the giving unit 12c refers to the table of the score setting table storage unit 13a (see FIG. 4) when the user's emotion determined by the known emotion recognition technology is “anger”. Then, it is determined that the score corresponding to “emotion (anger)” is “2”, and the evaluation classification “emotion (anger)” and the score “2” are stored in the table (FIG. 6) of the evaluation management table storage unit 13b. Is stored. Although the emotion recognition technique is not particularly limited, accuracy can be improved by using an emotion recognition technique using AI.

  Further, for example, the assigning unit 12c searches whether a change in emotion determined using a known emotion recognition technique is stored in the table (FIG. 4) of the score setting table storage unit 13a. As a result, when searching for data matching the change in the user's emotion, the assigning unit 12c assigns a corresponding score. Explaining with a specific example, when the user's emotion determined by a known emotion recognition technology changes from “anger” to “joy”, the giving unit 12c stores the table in the score setting table storage unit 13a (see FIG. 4), it is determined that the score corresponding to “emotion (anger → pleasure)” is “5”, and the evaluation classification “emotion (anger) is stored in the table (FIG. 6) of the evaluation management table storage unit 13b. → happy) "and the score" 5 ".

  Further, the giving unit 12c adds or subtracts a score according to the response of the operator specified from the data. For example, the assigning unit 12c determines whether a dialect or a foreign language is used when the operator responds, using a known voice recognition technology. Then, when it is determined that the dialect or the foreign language is used for the response used by the operator, the assigning unit 12c refers to the table (FIG. 5) of the score setting table storage unit 13a and assigns the corresponding score. For example, when the operator uses the dialect of Akita in response to the user, the assigning unit 12c refers to the table (FIG. 5) of the score setting table storage unit 13a and refers to the table (FIG. 5) of the evaluation management table storage unit 13b. FIG. 6) stores an evaluation classification “dialect (Akita) response” and a score “1”. Further, for example, when the operator responds using the NG word set in advance, the giving unit 12c refers to the table of the score setting table storage unit 13a (FIG. 5) and refers to the table of the evaluation management table storage unit 13b. The evaluation classification “NG word use” and the score “−1” are stored in the table (FIG. 6).

  Further, the assigning unit 12c increases the score according to the dialect or language used by the user specified from the data. For example, the providing unit 12c determines whether the user has used a dialect or a foreign language by using a known voice recognition technology. If the user speaks in Akita's dialect or English, the score is increased as if the user is in charge of a difficult case. For example, when the user uses the Akita dialect, the assigning unit 12c refers to the table of the score setting table storage unit 13a (FIG. 5) and stores the evaluation table in the evaluation management table storage unit 13b (FIG. 6). The classification “dialect (Akita)” and the score “1” are stored.

  Note that, in addition to the above example, the giving unit 12c may add or subtract a score when a predetermined condition is satisfied. For example, a score may be added when a response is performed according to a response scenario registered in advance or when a thank-you message is received from the user. Further, when a user who has been recognized as a claimer is recognized from the past response history, a score may be added. In addition, the giving unit 12c may perform score matching or score reduction based on various parameters such as matching of the conversation speed between the user and the operator, and the user's understanding skills, the learning status of the operator, and the response time. Good.

  The output unit 12d outputs an evaluation result of the operator based on the score given by the giving unit 12c. For example, when the output unit 12d detects that the response of the operator has ended, the output unit 12d obtains the score of the current case and the score of the latest month of the corresponding operator from the evaluation management table storage unit 13b. The output unit 12d totals the scores obtained in the current service case for each evaluation item and outputs the total as the evaluation result of the operator. In addition, the output unit 12d totals scores for a predetermined period for each evaluation item. After that, the output unit 12d causes the operator's terminal device 20 to display the score obtained in this case and the total score obtained in the last month. The output unit 12d may output the evaluation result to a terminal device or a storage unit in a center that stores the evaluation result to the evaluator of the operator, in addition to the terminal device 20 of the operator.

  For example, for each case, the output unit 12d displays, on the operator use system screen of the terminal device 20 of the operator, the score obtained in the current case, the added score and the reason obtained in the current case, Displays the deduction score and reason for the Item. Here, a display example of the evaluation result of the operator in this case will be described with reference to the example of FIG. FIG. 7 is a diagram illustrating a display example of the evaluation result of the operator in the current case. As illustrated in FIG. 7, the output unit 12d outputs the evaluation item, the obtained score, and the reason as the score obtained by the operator in the current case. In the example of FIG. 7, it is displayed that the content of the inquiry in this case is “technical inquiry / internet connection”, the score obtained is “2”, and the score obtained in the evaluation item related to emotion is “2”. Have been. Also, it is displayed that the score for the operator's response is “−1” and the reason for the score deduction is “Deduction (−1) due to use of NG word”.

  As described above, the output unit 12d totals the scores acquired in the current service case for each evaluation item, and displays the scores on the screen of the operator's terminal device 20. In addition, the output unit 12d totals items for which points have been added or deducted, and displays the totals on the screen of the operator's terminal device 20 together with the reasons.

  Further, a display example of the evaluation result of the operator in the latest month will be described using the example of FIG. FIG. 8 is a diagram showing a display example of the evaluation result of the operator in the last month. As illustrated in FIG. 8, the output unit 12d outputs an evaluation item, an obtained score, and a reason as a score obtained by the operator in the last month. In the example of FIG. 8, the score obtained for “technical inquiry / internet connection” is “84”, the score obtained for “technical inquiry / server setting” is “24”, and the “service inquiry / internet connection” is obtained. Is displayed as "251". By confirming such a screen, the operator can confirm that a score has been acquired by being in charge of many cases regarding “inquiry about service / internet”.

  Further, in the example of FIG. 8, it is displayed that the acquired score acquired in the evaluation item regarding the emotion is “35”. In the example of FIG. 8, the obtained score for the response of the operator is “40”, and the reason why the points were added or deducted is displayed. By checking such a screen, the operator can check what kind of response has led to the evaluation.

  Further, it may be possible to confirm the acquired score of the response for each evaluation classification. For example, as illustrated in FIG. 9, the output unit 12d tallies the score added or deducted by the response for each evaluation class, and obtains the acquired score for the response for each evaluation class on the screen of the terminal device 20 of the operator. To be displayed. As a result, it is possible for the operator to check which content of the inquiry is making a better response.

  Further, the output unit 12d may compare the total value of the scores of each operator in a certain period, perform ranking for each operator, and output ranking information as an evaluation result. For example, as illustrated in FIG. 10, the output unit 12d displays the relative evaluation level and the rank of the operator for each evaluation classification. The reference of this level shall be set in advance.

  For example, the output unit 12d calculates the rank of the operator for each evaluation axis using the total value of the scores for each evaluation axis acquired by the operator during a certain period (for example, one month). Then, the output unit 12d sets the level according to the ranking and displays the level for each evaluation classification on the operator or the evaluator screen of the operator. The output unit 12d may display the order when the order is high.

  The display timing of the screen may be displayed, for example, each time a response is made, or may be displayed when a display confirmation request is issued. In addition to displaying the relative evaluation results of the operators in the same center, the relative evaluation results of the operators may be displayed among a plurality of centers. That is, it is effective to display the relative evaluation results of the operators in the same center in a center having a relatively large number of operators, but the effect may be weak in a center having a small number of operators. In such a case, a multi-tenant contact center system may be assumed, and the relative evaluation result of the operator may be displayed across centers of a plurality of companies having the same or similar business type. As described above, the evaluation device 10 performs the evaluation and evaluation of the difficulty level not only for one company but also for a plurality of companies, so that the system can be evaluated highly even when the contact center is used in a relatively small contact center. It is possible.

  As described above, in the evaluation device 10 according to the first embodiment, the evaluation result described above can be checked by the operator or the evaluator of the operator, so that the motivation to improve the skill of the operator can be improved, and the evaluator of the operator can improve the motivation. It is possible to reduce the number of operations required for the evaluation.

[Processing procedure of evaluation device]
Next, an example of a processing procedure performed by the evaluation device 10 according to the first embodiment will be described with reference to FIGS. FIG. 11 is a flowchart illustrating an example of the flow of a score assignment process in the evaluation device according to the first embodiment. FIG. 12 is a flowchart illustrating an example of the flow of an evaluation result output process in the evaluation device according to the first embodiment.

  First, the flow of the score assignment process in the evaluation device will be described using the example of FIG. As illustrated in FIG. 11, the acquisition unit 12a of the evaluation device 10 acquires voice data or text data transmitted and received by the terminal device 20 of the operator in response to an inquiry from the user (Step S101).

  Subsequently, the classification unit 12b extracts a predetermined keyword from the data acquired by the acquisition unit 12a, and classifies the inquiry for each content based on the keyword (step S102). For example, the classification unit 12b performs a morphological analysis process and a keyword extraction process using a known text mining technique, and extracts a keyword included in the conversation. Then, the classification unit 12b refers to a table (for example, see FIG. 3) of the score setting table storage unit 13a, searches for an “inquiry class” that matches the extracted keyword combination, and sets the “inquiry class” that matches. Identify the corresponding “evaluation category”.

  The assigning unit 12c assigns a score indicating the difficulty level of the inquiry according to the content classified by the classifying unit 12b (step S103). For example, the assigning unit 12c refers to the table of the score setting table storage unit 13a to assign a score corresponding to the evaluation classification specified by the classification unit 12b. Further, the giving unit 12c adds or subtracts a score according to the emotion of the user and the response of the operator (Step S104). Then, the assigning unit 12c stores the score in the table (FIG. 6) of the evaluation management table storage unit 13b (Step S105).

  Next, the flow of the evaluation result output process in the evaluation device 10 will be described using the example of FIG. As illustrated in FIG. 12, when the output unit 12d of the evaluation device 10 detects that the response of the operator has ended (Yes at Step S201), the output unit 12d of the evaluation operator 10 stores the current time of the corresponding operator from the evaluation management table storage unit 13b. The score of the case and the score of the latest month are acquired (step S202).

  Then, the output unit 12d totals the scores of the latest month for each evaluation item (step S203), and displays the score obtained in the current case and the total score obtained in the latest month on the terminal device 20 of the operator. (Step S204).

(Effects of the First Embodiment)
The evaluation device 10 according to the first embodiment acquires data transmitted and received by a terminal device of an operator responding to an inquiry from a user, extracts a predetermined keyword from the acquired data, and makes an inquiry based on the keyword. Classify by content. Then, the evaluation device 10 assigns a score indicating the degree of difficulty of the inquiry according to the classified content, and outputs an evaluation result of the operator based on the assigned score. Therefore, it is possible to appropriately evaluate the response of the operator, including the difficulty level of the case. Thereby, the motivation of the operator can be maintained, and the turnover rate can be reduced.

  Further, the evaluation device 10 adds or subtracts scores according to the user's emotions or changes in emotions determined from the data. For this reason, the evaluation device 10 can add points, for example, when the user is angry and responds to a serious case, or when the user's anger can be calmed down. It is possible to evaluate objectively, including.

  Further, the evaluation device 10 adds or subtracts a score according to the response of the operator specified from the data. For this reason, the evaluation device 10 can add a score when an operator responds in a dialect or a foreign language, or can deduct a score when an NG word is said. It is possible to objectively evaluate the skills and response quality of the students.

  In addition, the evaluation device 10 adds a score according to the dialect or language used by the user specified from the data. For this reason, the evaluation device 10 can add a score, for example, when a user responds to an inquiry matter using a dialect or a foreign language, so that the degree of difficulty of the case when responding to a special case can be increased. It is possible to evaluate appropriately.

  In addition, when the response from the operator is completed, the evaluation device 10 totals the score of the present case and the score of the latest predetermined period for each evaluation item, and outputs the totaled score as the evaluation result of the operator. For this reason, the evaluation device 10 finely evaluates the difficulty of each response content and the degree of contribution of the operator, including the qualitative factors such as the technical difficulty of the response content and the emotional state of the user, and the operator himself / herself. Also, it is possible to provide appropriate feedback to the evaluator of the operator.

  Further, the evaluation device 10 compares the total value of the scores of each operator in a certain period, ranks each operator, and outputs rank information as an evaluation result. For this reason, in the evaluation device 10, it is possible to improve the motivation of the operator to improve the skill and to reduce the number of operations required for the evaluation by the evaluator of the operator.

(System configuration, etc.)
Each component of each device illustrated is a functional concept, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to the illustrated one, and all or a part of the distribution / integration may be functionally or physically distributed / arbitrarily in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Further, all or any part of each processing function performed by each device is realized by a CPU or GPU and a program analyzed and executed by the CPU or GPU, or as hardware by wired logic. Can be realized.

  Further, among the processes described in the present embodiment, all or a part of the processes described as being performed automatically can be manually performed, or the processes described as being performed manually can be performed. Can be automatically or completely performed by a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

(System configuration, etc.)
Each component of each device illustrated is a functional concept, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to the illustrated one, and all or a part of the distribution / integration may be functionally or physically distributed / arbitrarily in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Furthermore, all or any part of each processing function performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic.

  Further, among the processes described in the present embodiment, all or a part of the processes described as being performed automatically can be manually performed, or the processes described as being performed manually can be performed. Can be automatically or completely performed by a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

(program)
Further, it is also possible to create a program in which the processing executed by each device described in the above embodiment is described in a language executable by a computer. For example, it is also possible to create a program in which a process executed by the evaluation device 10 according to the embodiment is described in a computer-executable language. In this case, the same effects as in the above embodiment can be obtained by the computer executing the program. Further, the same processing as in the above embodiment may be realized by recording such a program on a computer-readable recording medium and reading and executing the program recorded on the recording medium by a computer.

  FIG. 13 is a diagram illustrating a computer that executes a program. As illustrated in FIG. 13, the computer 1000 has, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012 as illustrated in FIG. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090 as illustrated in FIG. The disk drive interface 1040 is connected to the disk drive 1100 as illustrated in FIG. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120 as illustrated in FIG. The video adapter 1060 is connected to, for example, a display 1130 as illustrated in FIG.

  Here, as exemplified in FIG. 13, the hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the above-described program is stored in, for example, the hard disk drive 1090 as a program module in which an instruction to be executed by the computer 1000 is described.

  The various data described in the above embodiment is stored as program data in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 as necessary, and executes various processing procedures.

  The program module 1093 and the program data 1094 relating to the program are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive or the like. . Alternatively, the program module 1093 and the program data 1094 related to the program are stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), or the like), and are connected via a network interface 1070. It may be read by the CPU 1020.

  The above embodiments and modifications thereof are included in the technology disclosed in the present application and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 10 Evaluation apparatus 11 Communication processing unit 12 Control unit 12a Acquisition unit 12b Classification unit 12c Assignment unit 12d Output unit 13 Storage unit 13a Score setting table storage unit 13b Evaluation management table storage unit 20A to 20F Terminal device 30 Internet 40A, 40B Company NW

Claims (9)

An acquisition unit configured to acquire data transmitted and received by a terminal device of an operator responding to an inquiry from a user;
A classification unit that extracts a predetermined keyword from the data acquired by the acquisition unit and classifies the inquiry for each content based on the keyword;
An assigning unit that assigns a score indicating the difficulty level of the inquiry, according to the content classified by the classifying unit,
An output unit that outputs an evaluation result of the operator based on the score assigned by the assigning unit.   2. The evaluation device according to claim 1, wherein the assigning unit adds or subtracts a score according to a user's emotion or a change in emotion determined from the data. 3.   2. The evaluation device according to claim 1, wherein the assigning unit adds or subtracts a score according to an operator's response specified from the data. 3.   2. The evaluation device according to claim 1, wherein the assigning unit adds a score according to a dialect or a language used by a user specified from the data. 3.   The output unit, when the response of the operator is completed, totals the score of the current case and the score of the latest predetermined period for each evaluation item, and outputs the totaled score as the evaluation result of the operator. The evaluation device according to claim 1.   2. The evaluation apparatus according to claim 1, wherein the output unit compares the total value of the scores of each operator in a certain period, ranks each operator, and outputs rank information as an evaluation result. The acquiring unit, when acquiring the voice data, using a voice recognition technology using AI, to separate the conversation between the user and the operator, and convert each conversation into text,
The evaluation apparatus according to claim 1, wherein the classification unit performs a morphological analysis process and a keyword extraction process using a text mining technique using an AI, and extracts a keyword from the text data. An evaluation method performed by the evaluation device,
An acquisition step of acquiring data transmitted and received by a terminal device of an operator in response to an inquiry from a user,
A classifying step of extracting a predetermined keyword from the data acquired in the acquiring step and classifying the inquiry for each content based on the keyword;
An assigning step of assigning a score indicating the degree of difficulty of the inquiry according to the content classified by the classifying step,
An output step of outputting an evaluation result of the operator based on the score assigned in the assigning step. An acquisition step of acquiring data transmitted and received by a terminal device of an operator in response to an inquiry from a user;
A classification step of extracting a predetermined keyword from the data acquired by the acquisition step, and classifying the inquiry for each content based on the keyword;
An assigning step of assigning a score indicating the degree of difficulty of the inquiry according to the content classified by the classifying step,
And an output step of outputting an evaluation result of the operator based on the score given by the giving step.

Images