JP2007257031A - Reception information evaluating program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve accuracy in evaluating an operator's skill even when the quantity of accumulative reception information is small. <P>SOLUTION: This reception information evaluating program makes a computer execute a method for evaluating evaluated-person simulated-reception information recording contents of simulated reception by an evaluated person for a predetermined inquiry item from a specific partner using reference person reception information recording contents of reception by a reference person for the predetermined inquiry item from the specific partner. A correction value for correcting partner feature quantity of the specific speaker included in the reference person reception information is calculated from the difference to the average value of the partner feature quantity of the other partner. A degree of matching between the reference person reception information and the evaluated-person simulated-reception information is calculated. The degree of matching is corrected by the correction value. Thus, the evaluation result of the evaluated-person simulated-reception information regarding the predetermined inquiry item is calculated. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a reception information evaluation program.

  2. Description of the Related Art Conventionally, call center operations are known in which an operator accepts and responds to orders, complaints, and other various inquiries from customers via a network such as a telephone. In such call center operations, the call center manager needs to provide individual education according to the operator's skills (for example, the ability to respond appropriately to customers and the ability to find and solve problem areas of inquiries). As a precondition, it is necessary for the call center manager to evaluate the skill of the operator.

  For this reason, as a method for evaluating the skill of an operator, for example, in the method disclosed in Patent Document 1, an exemplary operator (an operator with high skill) responds to an inquiry item by a specific customer. From the simulated response information in which the evaluation target operator is recorded in a simulated manner using the response information (for example, response record, response time, etc.) recorded, and the evaluation target operator records the simulated response. , Evaluate the skills of the operator being evaluated.

  Further, for example, in the method disclosed in Patent Document 2, the skill of the evaluation target operator is evaluated from the response information of the record in which the evaluation target operator actually records the inquiry item by a specific customer. is doing.

JP 2004-157253 A JP 2005-258551 A

  By the way, in the above-described conventional technology, there is a problem that the accuracy of evaluating the skill of the operator is lowered under a condition where the number of response information that can be accumulated is small.

  More specifically, in the method disclosed in Patent Document 1 described above, the operator to be evaluated is simulated using response information in which the content of an exemplary operator responding to an inquiry item by a specific customer is recorded. Because the evaluation information uses the simulated response information recorded by the operator who is the subject of the evaluation in a simulated manner, the evaluation result reflects the characteristics of this specific customer, and the operator's skill is evaluated. There is a problem that the accuracy to be reduced.

  Moreover, in the method disclosed in Patent Document 2 described above, it is necessary to use a large amount of response information in which the contents of the actual response of the operator to be evaluated for the inquiry item by a specific customer are used for the evaluation. For this reason, if there is a small amount of response information to be evaluated, the response information when responding to customers with abnormal characteristics will affect the overall evaluation, and the accuracy of evaluating operator skills will be reduced. There is a problem that it ends up.

  For this reason, for example, when a large amount of response information such as a VIP call center that responds to inquiries from some excellent customers cannot be accumulated, there is a problem that the skill of the operator cannot be accurately evaluated after all. is there.

  Therefore, the present invention has been made to solve the above-described problems of the prior art, and even if the number of response information that can be accumulated is small, the operator's skill can be reduced by excluding the characteristics of a specific customer. An object is to provide a reception information evaluation program capable of improving the accuracy of evaluation.

  In order to solve the above-described problems and achieve the object, the invention according to claim 1 uses the reference person response information in which the reference person responds to a predetermined inquiry item by a specific dialogue person, A response information evaluation program for causing a computer to execute a method for evaluating evaluator simulated response information in which the content of an evaluator responding to the predetermined inquiry item by the specific dialoguer is simulated, The correction value for correcting the interlocutor feature amount of the specific interlocutor included in the reference person response information is an average value of interlocutor feature amounts of other interrogators related to the predetermined inquiry item. A correction value calculating procedure for calculating from a deviation, a matching score calculating procedure for calculating a matching score between the reference person response information and the evaluator simulated response information, and the matching score calculated by the matching score calculating procedure And causing the computer to execute an evaluation result calculation procedure for correcting an evaluation result of the evaluator simulated reception information related to the predetermined inquiry item by correcting with the correction value calculated by the correction value calculation procedure. It is characterized by.

  Further, the invention according to claim 2 is a response that causes a computer to execute a method for evaluating an evaluated person performance response information in which a content of an evaluated person actually responding to a predetermined inquiry item by a specific dialogue person is recorded. An information evaluation program that uses the reference person response information that records the contents that the reference person has responded to a plurality of inquiry items. The degree of coincidence with the response information, and the input time ratio between the time required for the input of the addressee response information and the time required for the input of the reference person response information are calculated for each responder, and the calculated match degree and input A matching degree function deriving procedure for deriving a matching degree function for calculating a matching degree from the time ratio by using the input time ratio as a variable, and a specific dialogue person included in the evaluated person performance response information. A correction value calculation procedure for calculating a correction value for correcting the amount from a deviation from an average value of the conversation person feature amount of another conversation person related to the predetermined inquiry item, and correction by the correction value calculation procedure A time related to the input of the evaluator actual response information for which a value is calculated and a time related to input of the reference person response information related to the predetermined inquiry item by a dialogue person having the same correction value as the correction value A degree-of-matching calculation procedure for calculating a degree of matching between the evaluated person performance response information and the reference person response information by substituting the input time ratio into the degree-of-match function derived by the degree-of-match function derivation procedure, The degree of coincidence calculated by the degree of coincidence calculation procedure is corrected by the correction value calculated by the correction value calculation step, and the evaluation result of the person-to-be-evaluated actual response information related to the predetermined inquiry item is obtained. Characterized in that to execute the evaluation result calculation procedure to leave, to a computer.

  The invention according to claim 3 further includes a condition designation accepting procedure for accepting a condition designation for selecting evaluation subject performance response information to be evaluated in the above invention, wherein the correction value calculating procedure includes the condition designation It is characterized in that a correction value for correcting a conversation person feature amount of a specific conversation person included in the evaluated person performance response information selected by the condition designation accepted by the acceptance procedure is calculated.

  According to a fourth aspect of the present invention, in the above invention, the correction value calculating procedure calculates the correction value for each work phase of reception.

  According to a fifth aspect of the present invention, in the above invention, the correction value calculation procedure includes either the speech time obtained from voice data or the information relating to voice audibility as the dialogue feature amount. The correction value is calculated using two or more.

  According to the first aspect of the present invention, with respect to a predetermined inquiry item by a specific conversation person, using the reference person response information in which the content of the reference person responding to a predetermined inquiry item by a specific conversation person is recorded. A response information evaluation program for causing a computer to execute a method for evaluating simulated evaluator response information recorded by the evaluator in a simulated manner. Is calculated from the deviation from the average value of the interviewer's feature value obtained from the interaction information of other interactors related to the specified inquiry item, and the reference person's response information and the evaluator simulation The degree of coincidence with the response information is calculated, the degree of coincidence is corrected by the correction value, and the evaluation result of the evaluator simulated response information related to the predetermined inquiry item is calculated. When evaluating the simulated response information of the evaluator who recorded the simulated response of the operator to be evaluated using the response information), even if the number of response information that can be stored is small, As a result of correcting the feature amount, it is possible to improve the accuracy of evaluating the skill of the operator.

  According to the second aspect of the present invention, the computer is caused to execute a method for evaluating the evaluated person performance response information in which the content of the evaluated person actually responded to a predetermined inquiry item by a specific dialogue person is recorded. A response information evaluation program, in which the response information recorded by each of a plurality of respondents is recorded using the reference response information recorded by the reference person in response to a plurality of inquiry items, and the corresponding standard. The degree of coincidence with the customer service information and the input time ratio between the time for inputting the customer service information and the time for inputting the standard customer service information are calculated for each client, and the degree of coincidence and input time thus calculated are calculated. Deriving a degree-of-match function that calculates the degree of match from the ratio using the input time ratio as a variable, and calculating a correction value for correcting the interlocutor feature value of a specific interlocutor included in the evaluator performance response information. Calculated from the deviation from the average value of the interlocutor's feature value of other interrogators related to the inquiries item, the correction value is calculated and the same correction value is calculated for the time required to input the evaluation subject actual response information. By substituting the input time ratio with the time related to the input of the reference person response information related to the predetermined inquiry item by the interlocutor having the result into the derived degree-of-match function, the evaluation subject actual response information and the reference person response information The degree of coincidence is calculated, the degree of coincidence is corrected by a correction value, and the evaluation result of the evaluated person performance response information related to the predetermined inquiry item is calculated, so the contents of the actual response of the operator to be evaluated were recorded When evaluating customer-facing performance response information, the accuracy of evaluating operator skills can be improved as a result of the correction of specific customer features even under conditions where the number of response information that can be accumulated is small. It becomes ability.

  Further, according to the invention of claim 3, a condition designation for selecting the evaluated person performance response information to be evaluated is received, and the specific dialogue included in the evaluated person performance response information selected by the accepted condition specification The number of response information that can be accumulated when evaluating the actual response information of the evaluated person who recorded the actual response of the evaluation target operator is calculated. As a result, it is possible to further improve the accuracy of evaluating the skill of the operator as a result of selecting the evaluation subject performance response information to be evaluated by specifying the condition even under the condition where there are few.

  Further, according to the invention of claim 4, since the correction value is calculated for each work phase of reception, the feature quantity of a specific customer is determined for each work phase even under the condition that the number of response information that can be accumulated is small. As a result of the correction, it is possible to further improve the accuracy of evaluating the skill of the operator.

  According to the invention of claim 5, the correction value is calculated using one or more of the speech time obtained from the voice data or the information relating to the ease of hearing of the voice as the dialogue feature amount. Therefore, even when the number of response information that can be accumulated is small, the feature quantity of a specific customer is corrected, and as a result, the accuracy of evaluating the skill of the operator can be improved.

  Exemplary embodiments of a reception information evaluation program according to the present invention will be described below in detail with reference to the accompanying drawings. In the following, the main terms used in the embodiment, the outline and features of the response information evaluation apparatus according to the embodiment 1, the configuration and processing flow of the reception information evaluation apparatus, and the effects of the embodiment 1 will be described in order. Similarly to the first embodiment, the response information evaluation apparatus according to the second embodiment will be described in order, and finally another embodiment will be described.

[Explanation of terms]
First, main terms used in the following examples will be described. “Inquiry item (corresponding to“ inquiry item ”described in claims”) ”used in the following embodiments corresponds to a customer (“ interactive person ”described in claims) via a network such as a telephone. )), And the orders, complaints, and other various inquiries received by the call center operator are classified by item. For example, an operator who is undertaking a call center business related to a network service makes an inquiry such as “cannot connect to the network” or an inquiry such as “server is forcibly terminated”. Such inquiries are classified into common items between operators such as “net connection” and “server core dump” by the operator or the manager of the call center.

  The “evaluated person (corresponding to“ evaluated person ”described in the claims”) is an operator whose skill is evaluated by the manager of the call center. For skill evaluation, the representative operator (corresponding to the “reference person” described in the claims) records the response information of the customer (the “reference person reception information described in the claims”). "Corresponding"), the operator who is the evaluator responds in a simulated manner, and corresponds to the simulated response information (the "evaluator simulated response information" described in the claims). ) And the actual response information (corresponding to the “evaluator actual response information” described in the claims) that records the contents of the operator who actually received the customer. There are ways to do it. In the first embodiment, a method in which an operator who is a person to be evaluated responds in a simulated manner using the response information in which the content of an exemplary operator who responds to a customer is recorded, and the simulated response information in which the content is recorded is evaluated. Assuming The response information includes, for example, dialogue voice data that is voice data that records the content of dialogue between the customer and the operator, reception record that is an article that records the response content, response time information that records response time information, response time, Memo data that is a memo written as a material for creating a record.

[Outline and Features of Response Information Evaluation Apparatus According to Embodiment 1]
Then, the outline | summary and characteristic of the reception information evaluation apparatus based on Example 1 are demonstrated using FIG. FIG. 1 is a diagram for explaining the outline and features of the reception information evaluation apparatus according to the first embodiment.

  As described above, the response information evaluation apparatus according to the first embodiment uses the reference person response information in which the reference person responds to a predetermined inquiry item by the specific dialog person, and uses the reference person reception information. Evaluating the Evaluator's simulated response information that records the simulated response of the Evaluator for a given inquiry item, and for each inquiry item even under conditions where the number of response information that can be stored is small The main feature is to improve the accuracy of evaluating the skills of operators.

  Briefly explaining this main feature, as shown in FIG. 1, the response information evaluation apparatus according to the first embodiment is a model for a predetermined inquiry item (inquiry item Q10) by a specific dialogue person (customer A). Referencer response information in which the content of the referencer who is a correct operator is recorded is stored in advance. In the first embodiment, as reference person reception information, dialogue voice data, which is voice data in which a conversation between a customer and an operator is recorded, reception record, which is an article in which the reception content is recorded, and reception time information are recorded. Response time information and memo data, which is a memo written as a material for creating a response record, are stored in advance, but in FIG. 1, the dialogue voice data and other response records are roughly divided into two.

  First, the response information evaluation apparatus according to the first embodiment uses reference person response information in which contents are recorded by responding to a reference person with respect to a predetermined inquiry item (inquiry item Q10) by a specific dialogue person (customer A). Then, the evaluator simulated response information in which the content is recorded by responding to the inquired person in response to the inquiry item Q10 by the customer A is acquired (see (1) in FIG. 1).

  Next, the response information evaluation apparatus sets an average value of the interlocutor feature values of other interrogators related to the inquiry item Q10 as a correction value for correcting the interlocutor feature amount of the customer A included in the reference person response information. It is calculated from the deviation from the value (see (2) in FIG. 1).

  Then, the reception information evaluation apparatus calculates the degree of coincidence between the reference person reception information and the evaluated person simulated reception information (see (3) in FIG. 1).

  Subsequently, the response information evaluation apparatus calculates the evaluation result of the evaluator simulated response information related to the inquiry item Q10 by correcting the calculated degree of coincidence with the calculated correction value (see (4) in FIG. 1). reference).

  For this reason, the response information evaluation apparatus according to the first embodiment uses the reference person response information (response information handled by the model operator) to record the contents of the simulated response by the operator to be evaluated. When evaluating simulated response information, even when the number of response information that can be accumulated is small, the feature quantity of a specific customer is corrected. As a result, the accuracy of evaluating the skill of the operator can be improved.

[Configuration of the service information evaluation apparatus according to the first embodiment]
Next, the configuration of the reception information evaluation apparatus according to the first embodiment will be described with reference to FIGS. FIG. 2 is a block diagram illustrating a configuration of the reception information evaluation apparatus according to the first embodiment, FIG. 3 is a diagram for explaining a screen image for inputting reception information (response recording), and FIG. FIG. 5 is a diagram for explaining a screen image for each inquiry content, FIG. 5 is a diagram for explaining an output example of an evaluation result of the evaluator simulated response information, and FIGS. 6 and 7 are reference user responses; 8 is a diagram for explaining the information storage unit, FIG. 8 is a diagram for explaining the to-be-evaluated simulated response information storage unit, and FIGS. 9 to 15 explain the response information storage unit for each inquiry item. FIG. 16 is a diagram for explaining the correction value storage unit, FIG. 17 is a diagram for explaining the coincidence degree storage unit, and FIG. 18 is a diagram illustrating the response information reproduction logging unit. FIG. 19 to FIG. 22 illustrate the correction value calculation unit. A diagram for FIG. 23 and FIG. 24 is a diagram for explaining a coincidence degree calculation unit, FIGS. 25 to 28 are diagrams for explaining the evaluation results calculator.

  As shown in FIG. 2, the reception information evaluation apparatus 10 mainly includes an input unit 11, an output unit 12, an input / output control IF unit 13, a storage unit 20, and a control unit 30.

  The input unit 11 is an input unit that inputs data used for various types of processing by the control unit 30 and operation instructions for performing various types of processing using a microphone, a keyboard, a storage medium, or communication. For example, reference person response information that records the content of the reference person's response to the inquiry item by the interlocutor, or an evaluator simulation that records the content of the evaluation person's response to the inquiry item by the dialog person Inputs response information and operation instructions for evaluating the evaluator simulated response information.

  To give a specific example, when the reference person or the person to be evaluated responds to the inquiry item by the conversation person in the reception information evaluation apparatus 10, as shown in FIG. According to the screen output to the monitor by the unit 12, information such as the affiliation code, company name, organization name, telephone number, name, etc. of the conversation person, inquiry contents, correspondence contents, correspondence results, etc. are input. The reference person reception information input by the input unit 11 is stored in a reference person reception information storage unit 21 to be described later, and the evaluator simulated reception information input by the input unit 11 is stored in the evaluator simulated reception information storage. Stored in the unit 22.

  The output unit 12 is an output unit that outputs the results of various processes by the control unit 30 and operation instructions for performing various processes to a monitor, a printer, a speaker, or the like. For example, a screen for allowing the input unit 11 to input response information that the reference person or the evaluated person has responded to an inquiry item by the conversation person, or an evaluated person simulation that records the contents that the evaluated person has simulated Output the evaluation result of the response information.

  As a specific example, as described above, when the reference person or the person to be evaluated responds to the inquiry item by the conversation person in the reception information evaluation apparatus 10, as shown in FIG. Outputs a screen for allowing the input unit 11 to input information such as the affiliation code, company name, organization name, telephone number, name, etc., inquiry content, response content, response result, etc. As shown in FIG. 4, the output unit 12 outputs a screen for input by the input unit 11 for each inquiry content (for example, question content, complaint content, etc.) from the interlocutor. Further, as shown in FIG. 5, the evaluation result of the evaluator simulated reception information is output in association with the evaluator name (“evaluator_a”) and the evaluation result (“30”).

  The input / output control IF unit 13 is means for controlling data transfer between the input unit 11 and the output unit 12, and the storage unit 20 and the control unit 30.

  The storage unit 20 is a storage unit that stores data used for various types of processing by the control unit 30, and as particularly closely related to the present invention, as shown in FIG. The evaluator simulated response information storage unit 22, the inquiry item-specific response information storage unit 23, the correction value storage unit 24, the coincidence degree storage unit 25, and the evaluation result storage unit 26 are provided.

  Among the storage units 20, the reference person reception information storage unit 21 is necessary for the receptionee to respond in a simulated manner to a predetermined inquiry item by a specific dialogue person in the reception information evaluation apparatus 10. This is storage means for storing reference person reception information input by the input unit 11. Specifically, dialogue voice data, response record, response time information, memo data, and the like, which are reference person response information in which the reference person responds to an inquiry item by the dialog person, are stored. For example, as shown in FIG. 6, the reference person reception information storage unit 21 stores information such as the affiliation code, company name, organization name, telephone number, name, etc. of the conversation person, inquiry contents, and response contents as reception records. . For example, as shown in FIG. 7, the reference person reception information storage unit 21 stores memo data, which is a memo written as a material for which a reference record is created by the reference person.

  The evaluator simulated response information storage unit 22 is storage information that is evaluated by the response information evaluation apparatus 10 and stores the evaluator simulated response information input by the input unit 11. Specifically, dialogue voice data, response records, response time information, memo data, and the like, which are response information of the evaluator that records the content of the evaluator's response to the inquired item by the interrogator, are stored. . For example, as shown in FIG. 8, the evaluator simulated reception information storage unit 22 stores information such as the affiliation code, company name, organization name, telephone number, name, etc. of the conversation person, inquiry content, response content, etc. Remember.

  In the inquiry item-specific response information storage unit 23, the response information stored in the reference person response information storage unit 21 and the evaluator simulated response information storage unit 22 is given information and classified by the response information reproduction logging unit 31 described later. This is a storage means for storing the object. Specifically, an operation log and operation time information are given by a response information reproduction logging unit 31 described later, and the response information classified according to the contents of the inquiry is stored.

  For example, the reference person reception information storage unit 21 stores dialogue voice data between the “customer A” who is a dialogue person and “reference person_a” which is a reference person as reference person reception information. As shown in FIG. 9, the storage unit 23 stores the dialogue voice data between the customer A and the reference person_a acquired as separate dialogue voice data by the reception information reproduction logging unit 31. Further, for example, the reference person reception information storage unit 21 stores the memo data shown in FIG. 7 as reference person reception information, but the inquiry item-specific reception information storage unit 23 stores the reference person as shown in FIG. Is stored in the reception information reproduction logging unit 31 in association with the order of information and time information.

  Further, for example, the reference person reception information storage unit 21 stores the reception record shown in FIG. 6 as the reference person reception information, but the inquiry item-specific reception information storage unit 23 stores the reception information as shown in FIG. The reproduction logging unit 31 associates the operation content, operation time, and dialogue voice data, and further, a response number (for example, “A0001”) to be managed by the inquiry item-specific response information storage unit 23 is given. The information to which the performance flag indicating that it is the reception information in which the content of the reference person actually responding to the inquiry item is recorded is stored. Further, for example, the inquiry item-specific response information storage unit 23 stores the response information shown in FIG. 11 categorized by inquiry item (for example, “[question] net connection”) as shown in FIG. . As shown in FIG. 13, the inquiry item response information storage unit 23 stores various types of response information stored in association with the response number (eg, “A0001”) as a key.

  On the other hand, for example, the person-to-be-evaluated simulated response information storage unit 22 stores the response record shown in FIG. 8 as the person-evaluated simulated response information. In addition, the operation information, the operation time, the work phase information, and the dialogue voice data are associated with each other by the response information reproduction / logging unit 31, and the response number (for example, “A0002” or the like for management in the inquiry item specific response information storage unit 23) ) And a test flag indicating that it is response information in which the content of the subject to be evaluated is recorded in response to the inquiry item by the interrogator is stored. Further, for example, the inquiry item-specific response information storage unit 23 classifies and stores the response information shown in FIG. 14 by inquiry item (for example, “[question] net connection”) as shown in FIG. To do.

  The correction value storage unit 24 is a storage unit that stores a correction value calculated by a correction value calculation unit 32 described later. Specifically, a correction value for correcting the dialogue feature amount of a specific dialogue person included in the reference person response information is stored. For example, as illustrated in FIG. 16, the correction value storage unit 24 stores the average value of the conversation person feature amount (“average conversation person feature amount”) and the conversation person feature amount of the customer A (“interaction person feature amount”). ) And the correction value of customer A (“correction value”). In the first embodiment, it is assumed that the inquiry is composed of four phases: “reception phase”, “inquiry content grasping phase”, “problem factor identification phase”, and “problem solving phase”. Therefore, the correction value is also stored for each of these work phases.

  The coincidence degree storage unit 25 is a storage unit that stores the coincidence degree calculated by the coincidence degree calculation unit 33 described later. Specifically, the reference person response information that records the contents of the reference person's response to a predetermined inquiry item by a specific dialogue person, and the evaluator simulates the predetermined inquiry item by a specific conversation person The degree of coincidence with the evaluator simulated response information in which the content corresponding to the response is recorded is stored. For example, as shown in FIG. 17, the coincidence degree storage unit 25 includes reference person response information (“value of the response record entered by the reference person”) and evaluator simulated response information (“response record entered by the evaluator”). And the degree of coincidence (“document content coincidence degree”) with the value “)”. In the first embodiment, the degree of coincidence is stored and the input time ratio is also stored for the purpose of deriving the degree of coincidence function described in the second embodiment to be described later. Although the ratio (“input time ratio”) is shown as the stored content, it is not always necessary to store the input time ratio for the reception information evaluation apparatus 10 according to the first embodiment. Only the degree of coincidence may be stored.

  The evaluation result storage unit 26 is a storage unit that stores an evaluation result calculated by an evaluation result calculation unit 34 described later. Specifically, the evaluation result of the to-be-evaluated simulated response information related to a predetermined inquiry item in which the degree of coincidence calculated by the degree of coincidence calculating unit 33 is corrected by the correction value calculated by the correction value calculating unit 32 is stored. . For example, as shown in FIG. 5, the evaluated person name (“evaluated person name”) and the evaluation result of the evaluated person simulated response information (“evaluated person testing skill”) are stored in association with each other.

  Here, returning to FIG. 2, the control unit 30 is a control means for controlling the reception information evaluation apparatus 10 to execute various processes. Particularly, as closely related to the present invention, as shown in FIG. 2. In addition, a reception information reproduction logging unit 31, a correction value calculation unit 32, a matching degree calculation unit 33, and an evaluation result calculation unit 34 are provided.

  In the control unit 30, the reception information reproduction logging unit 31 is a means for adding and classifying information to the reception information stored in the reference person reception information storage unit 21 and the evaluator simulated reception information storage unit 22. is there. Specifically, an operation log and operation time information are given to the response information stored in the reference person response information storage unit 21 and the evaluator simulated response information storage unit 22, and classified according to the content of the inquiry. It is stored in the reception information storage unit 23.

  For example, as shown in FIG. 18, the reception information reproduction logging unit 31 associates operation contents, operation time, and dialogue voice data, and also receives a response number (for example, “ A0001 etc. ") is assigned, and further classified by inquiry item (for example," [question] net connection "etc.). Here, the association of the response time will be described with reference to FIG. 18. The response information reproduction logging unit 31 has a time (0 [sec]) when the cursor is fixed to the affiliated code input form and a time when the cursor is moved ( 8 [sec]), the response time for the belonging code is associated with 8 seconds. Similarly, the reception information reproduction logging unit 31 inputs a memo (“net connection”) from the time when the cursor is fixed to the memo (0 [sec]) and the time when the cursor is moved (6 [sec]). Assume that the response time for is 6 seconds.

  The correction value calculation unit 32 is a means for calculating a correction value for correcting the feature amount of a specific conversation person. More specifically, a correction value for correcting a conversation person feature amount of a specific conversation person included in the reference person reception information stored in the inquiry information for each inquiry item 23 is set in accordance with a predetermined inquiry item. Is calculated from the deviation from the average value of the conversation person feature value of the conversation person and stored in the correction value storage unit 24. The correction value calculation unit 32 corresponds to the “correction value calculation procedure” recited in the claims.

  Here, the response information evaluation apparatus 10 according to the first embodiment uses various response information as a response work phase (for example, “reception phase”, “inquiry content grasping phase”, “problem factor identification phase”, “problem solving phase”). Etc.) and evaluate each. For this reason, the correction value calculation unit 32 according to the first embodiment uses the dialog voice data, the response time information, the operation log, and the operation time information among the reference person response information that the reference person has responded to. And the work phase are estimated, and the talker's utterance time for each work phase is calculated. A method for estimating the correspondence between the voice of the dialogue voice data and the work phase will be described with reference to FIG. 19. First, the voice data by the customer is associated with the operation log by the reference person. Then, it is estimated that the voice data by the customer starts from the “reception phase”. Next, it can be estimated that the utterance after the operation log in which the screen transition button is pressed in the “reception phase” is the “inquiry content grasping phase”. Similarly, it can be estimated that the “problem factor identification phase” is from the utterance after the operation log in which the screen transition button is pressed in the “inquiry content grasping phase”. In the “problem factor identification phase”, for example, an operation such as searching for knowledge related to an inquiry is performed, so an operation log for pressing the search button appears. In the “problem solving phase”, since the customer is recalled, an operation log for pressing the recall button appears.

  Next, as shown in FIG. 20, the correction value calculation unit 32 sums up the customer utterance time for each of the “reception phase”, “inquiry content grasping phase”, “problem factor identification phase”, and “problem solving phase”. These are the interlocutor feature values for each work phase. Then, as shown in FIG. 21, the correction value calculation unit 32 calculates an average value of the conversation person feature amount of the other conversation person relating to the predetermined inquiry item, and the dialogue of the specific conversation person for each work phase. To calculate the difference between the person's feature value and the average value of other talker's feature values, and to use the Gaussian error function to correct the talker feature value of the specific talker included in the reference person's response information The correction value is calculated as shown in FIG. 22 and stored in the correction value storage unit 24.

  The degree of coincidence calculation unit 33 is a means for calculating the degree of coincidence between the reference response information and the evaluator simulated response information. Specifically, the degree of coincidence between the reference response information stored in the inquiry item-specific response information storage unit 23 and the evaluator simulated response information is calculated and stored in the coincidence degree storage unit 25. The coincidence calculation unit 33 corresponds to the “coincidence calculation procedure” described in the claims.

  Here, the degree-of-match calculation unit 33 according to the first embodiment extracts the reference person response information and the evaluator simulated response information input to the same question number from the inquiry item-specific response information storage unit 23, as shown in FIG. Then, the degree of coincidence is calculated using a vector space model. That is, all combinations of paths obtained by dictionary lookup for the input character string are obtained, and the path with the maximum probability value is selected (morphological analysis). Then, the degree of coincidence is obtained by obtaining the angle of the vector of the set L of morphemes of the reference person and the person to be evaluated. In the first embodiment, as shown in FIG. 24, the degree of coincidence is calculated and the input time ratio is also calculated for the purpose of being used for deriving the degree of coincidence function described in the second embodiment described later. In the reception information evaluation apparatus 10 according to the first embodiment, it is not always necessary to calculate the input time ratio, and the coincidence degree calculation unit 33 may calculate only the coincidence degree.

  The evaluation result calculation unit 34 is a means for calculating the evaluation result of the evaluator simulated reception information. Specifically, using the correction value calculated by the correction value calculation unit 32 and the degree of coincidence calculated by the degree of coincidence calculation unit 33, the evaluation result of the evaluator simulated response information related to a predetermined inquiry item is obtained. The calculation result is stored in the evaluation result storage unit 26. The evaluation result calculation unit 34 corresponds to the “evaluation result calculation procedure” described in the claims.

  Here, the evaluation result calculation part 34 in Example 1 calculates the evaluation result of the to-be-evaluated person simulated reception information by the mathematical formula shown in FIG. First, the left side I of the expression (1-1) in FIG. 25 represents the evaluation result of the evaluator simulated response information, and the right side E (i) is the conversation person feature amount in the work phase to which the question q (i) belongs. H represents a value obtained by dividing the total score G of the reference person's score by the perfect score V. In addition, the right side C (jk_i) of the expression (1-2) in FIG. 25 represents the degree of coincidence of the response records for the question q (i) input by the reference person and the evaluator. Also, the formula (1-3) in FIG. 25 is a formula for calculating the total point G of the above-mentioned reference person's points, and is calculated using the dialogue feature amount.

  Next, Equations (1-4) to (1-8) in FIG. 26 represent the evaluation results of the evaluator simulated response information represented by Equation (1-1) in FIG. 25 for each work phase. It is a thing. The expression (1-4) in FIG. 26 is an evaluation result of the “acceptance phase”, and indicates that the question numbers i are 1 to 5. Also, the expression (1-5) in FIG. 26 is an evaluation result of the “inquiry content grasping phase” and indicates that the question number i is 6. Expression (1-6) in FIG. 5) This is a modification of the equation. Also, the expression (1-7) in FIG. 26 is the evaluation result of the “problem factor identification phase”, and indicates that the question number i is 7-9. Also, the expression (1-8) in FIG. 26 is an evaluation result of the “problem solving phase”, and indicates that the question number i is 10.

  Further, the expression (1-9) in FIG. 27 is an expression obtained by concretely developing the expression (1-3) in FIG. 25, and the question number i of the “reception phase” is 1 to 5. Since the coefficient of the dialogue person feature amount E (reception_A0001) in the “reception phase” is 5 and the question number i in the “inquiry content grasping phase” is 6, the conversation person feature amount E in the “inquiry content grasping phase”. The coefficient of (Understanding_A0001) is 1, and since the question number i of the “problem factor identification phase” is 7 to 9, the coefficient of the conversation person feature E (specific_A0001) in the “problem factor identification phase” Is 3, and the question number i of the “problem solving phase” is 10, so that the coefficient of the dialogue feature amount E (solution_A0001) in the “problem solving phase” is 1.

  28, the evaluation result I (reception_A0001) of the “reception phase”, the evaluation result I (recognition_A0001) of the “inquiry content grasping phase”, the “problem factor identification phase” The sum of the evaluation result I (specific_A0001) and the evaluation result I (solution_A0001) of the “problem solving phase” multiplied by the normalization coefficient (in the expression (1-10), the maximum score V is 100 Is the evaluation result of the person-to-be-evaluated simulated response information.

[Processing by Response Information Evaluation Apparatus in Embodiment 1]
Next, the process by the reception information evaluation apparatus in Example 1 is demonstrated using FIGS. 29-35. FIG. 29 is a diagram for explaining the flow of information related to the service information in the first embodiment, FIG. 30 is a flowchart showing the procedure of the service information evaluation process in the first embodiment, and FIG. FIG. 32 is a flowchart showing the procedure of the reception information reproduction logging process in the first embodiment, and FIG. 33 is the flowchart of the correction value calculation process in the first embodiment. FIG. 34 is a flowchart showing the procedure of the coincidence degree calculation process in the first embodiment, and FIG. 35 is a flowchart showing the procedure of the evaluation result calculation process in the first embodiment.

[Flow of information related to reception information in Embodiment 1]
First, as shown in FIG. 29, as reference person response information in which a reference person has responded to a predetermined inquiry item by a specific dialog person, specifically, dialogue voice data (reference person), response information is recorded. Recording (reference person), reception time information (reference person), and memo data (reference person) are input to the reception information reproduction logging unit 31. Among these pieces of reception information, dialogue voice data (reference person) and memo data (reference person) are output by the reception information reproduction logging unit 31 in order to make the evaluated person respond in a simulated manner.

  Next, using the dialogue voice data (reference person) and memo data (reference person) output by the reception information reproduction logging unit 31, the person to be evaluated simulates a predetermined inquiry item by a specific talker. The response record (evaluated person) and the response time information (evaluated person) are input to the response information reproduction logging unit 31 among the evaluated person simulated response information in which the response contents are recorded.

  The reference person response information and the evaluator simulated response information are given operation logs and operation time information by the response information reproduction logging unit 31, classified for each inquiry item, and stored in the inquiry item-specific response information storage unit 23. The

  Of the reception information stored in the inquiry information storage unit 23 by inquiry item, dialogue voice data (reference person), reception record (reference person), reception time information (reference person), operation log (reference person), and operation time Information (reference person) is input to the correction value calculation unit 32, reception record (reference person and evaluated person), response time information (reference person and evaluated person), operation log (reference person and evaluated person), and Operation time information (reference person and person to be evaluated) is input to the coincidence degree calculation unit 33.

  In the correction value calculation unit 32, a correction value for correcting a conversation person feature amount of a specific conversation person included in the reference person reception information is obtained from the inputted reception information as another conversation related to a predetermined inquiry item. It is calculated from the deviation from the average value of the talker's feature value. On the other hand, the coincidence calculation unit 33 calculates the coincidence and input time ratio between the reference response information and the evaluator simulated response information from the input response information. Then, the evaluation result calculation unit 34 corrects the calculated degree of coincidence with the calculated correction value, calculates the evaluation result of the evaluator simulated response information related to the predetermined inquiry item, and the evaluation result storage unit 26 Remember me. In the first embodiment, the degree of coincidence calculation unit 33 calculates the degree of coincidence and also calculates the input time ratio for the purpose of being used for deriving the degree of coincidence function described in Example 2 described later. In the reception information evaluation apparatus 10 according to the first embodiment, it is not always necessary to calculate the input time ratio, and the coincidence degree calculation unit 33 may calculate only the coincidence degree.

[Response information evaluation process]
Next, as shown in FIG. 30, the response information evaluation apparatus 10 according to the first embodiment accepts input of reference person response information in which the reference person responds to a predetermined inquiry item by a specific dialogue person. And stored in the reference person reception information storage unit 21 (step S3001).

  Then, the reception information evaluation apparatus 10 adds an operation log and operation time information to the information stored in the reference person reception information storage unit 21 in the reception information reproduction logging unit 31, classifies the information according to the inquiry item, and makes an inquiry. Stored in the item-specific response information storage unit 23 (step S302).

  Next, the reception information evaluation apparatus 10 uses the reference person reception information to record the evaluation person simulated reception information in which the evaluation person records the contents of the simulation in response to a predetermined inquiry item by a specific dialogue person. When the evaluation instruction is received (Yes at step S3003), the response information reproduction logging unit 31 adds an operation log and operation time information to the information stored in the evaluator simulated response information storage unit 22, and classifies the information for each inquiry item. Then, the inquiry information is stored in the inquiry information storage unit 23 (step S3004).

  Subsequently, the response information evaluation apparatus 10 obtains a correction value for correcting the interlocutor feature amount of a specific interrogator included in the reference person response information stored in the inquiry item-specific response information storage unit 23 as a predetermined inquiry. It is calculated from the deviation from the average value of the other conversation person's feature value related to the item, and is stored in the correction value storage unit 24 (step S3005).

  The response information evaluation apparatus 10 calculates the degree of coincidence between the reference person response information stored in the inquiry item-specific response information storage unit 23 and the evaluator simulated response information, and stores it in the coincidence degree storage unit 25 (step). S3006).

  Subsequently, the response information evaluation apparatus 10 corrects the coincidence degree stored in the coincidence degree storage unit 25 with the correction value stored in the correction value storage unit 24, and evaluates the simulated person to be evaluated according to a predetermined inquiry item. An evaluation result of information is calculated and stored in the evaluation result storage unit 26 (step S3007).

  In step S3003, when the evaluation instruction for the evaluator simulated reception information is not received (No in step S3003), the reception information evaluation apparatus 10 stands by until the evaluation instruction is received. In the first embodiment, the case where the matching degree calculation process (step S3006) is performed following the correction value calculation process (step S3005) has been described. However, the reception information evaluation apparatus 10 performs the correction following the matching degree calculation process. Even when the value calculation process is performed, the correction value calculation process and the coincidence degree calculation process may be performed simultaneously.

[Responding information playback logging process]
Here, with reference to FIG. 31, the response information reproduction logging process in the first embodiment (step S3002 in FIG. 30) will be described in detail. The response information evaluation apparatus 10 performs a response to a predetermined inquiry item by a specific conversation person. Information received by the reference person is received and stored in the reference person reception information storage unit 21. In the reception information reproduction logging unit 31, the information stored in the reference person reception information storage unit 21 is stored. Is received (step S3101-1). Specifically, the input of dialogue voice data, reception record, and memo data among the reference person reception information is accepted.

  At the same time, the reception information reproduction logging unit 31 measures the reception time (step S3101-2), logs the operation log (step S3101-3), and measures the operation time (step S3101-4).

  Next, the reception information reproduction logging unit 31 uses the reference person reception information of the dialog voice data, reception recording, and memo data received in step S3101-1 and the reception time information measured in step S3101-2. Thus, a measurement time is assigned to each predetermined question of the response record (step S3102-1).

  At the same time, the reception information reproduction logging unit 31 assigns an operation time for each action of the operation log using the operation log logged in step S3101-3 and the operation time measured in step S3101-4. (Step S3102-2).

  Subsequently, the reception information reproduction logging unit 31 uses the measurement time given for each question of the response recording in step S3102-1 and the operation time given for each action of the operation log in step S3102-2. An operation log and operation time information are added to the reception information (step S3103).

  Then, the reception information reproduction logging unit 31 classifies the reception information given the operation log and the operation time information in Step 3103 for each inquiry item (Step S3104).

[Responding information playback logging process]
Next, the response information reproduction logging process (step S3004 in FIG. 30) in the first embodiment will be described in detail with reference to FIG. 32. The response information evaluation logging device 31 performs a specific dialogue in the response information reproduction logging unit 31. Of the reference person response information in which the reference person has responded to a predetermined inquiry item by the user, the dialogue voice data is reproduced (step S3201-1), and the memo data is displayed in time order according to the reproduction time. (Step S3201-2).

  At the same time, the reception information evaluation apparatus 10 accepts input of the evaluator simulated response information in which the content of the evaluator is simulated using the reference person response information, and receives the evaluator simulated response information storage unit. The reception information reproduction logging unit 31 receives the input of information stored in the evaluated person reception information storage unit 22 (step S3201-3).

  At the same time, the reception information reproduction logging unit 31 measures the reception time (step S3201-4), logs the operation log (step S3201-5), and measures the operation time (step S3201-6).

  Next, the reception information reproducing / logging unit 31 uses the reception information of the reception record received in step S3201-3 and the reception time information measured in step S3201-4 to determine the reception recording in advance. A measurement time is assigned to each question (step S3202-1).

  At the same time, the reception information logging unit 31 gives an operation time for each action of the operation log using the operation log logged in step S3201-5 and the operation time measured in step S3201-6 ( Step S3202-2).

  Subsequently, the reception information reproduction logging unit 31 uses the measurement time given for each question of the response recording in step S3202-1 and the operation time given for each action of the operation log in step S3202-2. An operation log and operation time information are added to the reception information (step S3203).

  Then, the reception information reproduction logging unit 31 classifies the reception information given the operation log and the operation time information in step S3203 for each inquiry item (step S3204).

[Correction value calculation processing]
Next, the correction value calculation process (step S3005 in FIG. 30) in the first embodiment will be described in detail with reference to FIG. 33. The reception information evaluation apparatus 10 uses the correction value calculation unit 32 to change the reference person reception information. An input is accepted (step S3301).

  Then, the correction value calculation unit 32 specifies the work phase by combining the operation log on the time axis of the dialog voice data in the reference person response information received in step S3301 (step S3302).

  Subsequently, the correction value calculation unit 32 totals the response time for each work phase identified in step S3302 (step S3303).

  Then, the correction value calculation unit 32 determines whether or not dialogue voice data remains after the identified work phase (step S3304). If the dialogue voice data remains (Yes in step S3304), the correction value calculation unit 32 again determines the dialogue voice data. The operation log is specified on the time axis to identify the work time phase. If no dialogue voice data remains (No at Step S3304), the correction value calculation unit 32 receives the input of the reception information recorded at the time of reception with other customers with the same inquiry item (Step S3305).

  Next, the correction value calculation unit 32 identifies the work phase by combining the operation log on the time axis of the dialog voice data in the response information received in step S3305 (step S3306).

  Subsequently, the correction value calculation unit 32 totals the response time for each work phase specified in step S3306 (step S3307).

  Then, the correction value calculation unit 32 determines whether or not the dialog voice data remains after the identified work phase (step S3308). If the dialog voice data remains (Yes in step S3308), the correction value calculation unit 32 again determines the dialog voice data. The operation log is specified on the time axis to identify the work time phase. If no dialogue voice data remains (No at step S3308), the correction value calculation unit 32 determines whether there is customer service information for which the work phase has not been specified (step S3309). If there is unfinished reception information (Yes at step S3309), processing for specifying the work time phase is performed again by matching the operation log on the time axis of the dialog voice data. If there is no response information that has not been specified (No at Step S3309), the correction value calculation unit 32 calculates an average value of the response time for each work phase (Step S3310).

  Next, the correction value calculation unit 32 calculates the difference between the average value calculated in step S3310 and the response time calculated in step S3303 (step S3311).

  Then, the correction value calculation unit 32 calculates a correction value using a Gaussian error function (step S3312).

[Matching degree calculation process]
Next, with reference to FIG. 34, the degree-of-matching calculation process in the first embodiment (step S3006 in FIG. 30) will be described in detail. The input of the evaluator simulated response information is accepted (step S3401).

  Then, the coincidence calculation unit 33 extracts a document for each question from the response record of the reference person response information and the response record of the evaluator simulated response information, and calculates the coincidence using the vector space model (step S3402). .

  Subsequently, the degree of coincidence calculation unit 33 calculates the ratio of the time taken for each question input from the response record of the reference person response information and the response record of the evaluator simulated response information (step S3403).

  Then, the coincidence calculation unit 33 calculates the coincidence in step S3402, and attaches a calculation completion flag to the question (response record) for which the input time ratio has been calculated in step S3403 (step S3404).

  Next, the coincidence calculation unit 33 determines whether there is a question without a calculation completion flag (step S3405). If there is a question (Yes in step S3405), the reference person response information is again displayed. A document for each question is extracted from the response record and the response record of the evaluator simulated response information, and the degree of coincidence is calculated using a vector space model. When there is no question (No at Step S3405), the coincidence calculation unit 33 records the ratio between the coincidence and the input time (Step S3406).

  In the first embodiment, the case in which the degree of coincidence is calculated and the input time ratio is calculated has been described. However, as the reception information evaluation apparatus 10 according to the first embodiment, it is not essential to calculate the input time ratio. Only the degree of coincidence may be calculated.

[Evaluation result calculation process]
Next, the evaluation result calculation process (step S3007 in FIG. 30) in the first embodiment will be specifically described with reference to FIG. 35. The reception information evaluation apparatus 10 is created by the reference person in the evaluation result calculation unit 34. The reception record and the input of the correction value and the matching degree for each work phase are received (step S3501).

  Then, the evaluation result calculation unit 34 calculates the test score of the reference person using the correction value received in step S3501 (step S3502).

  Subsequently, the evaluation result calculation unit 34 calculates a normalization coefficient using the test score of the reference person calculated in step S3502 (step S3503).

  Then, the evaluation result calculation unit 34 calculates the evaluation result of the evaluator simulated reception information (step S3504).

[Effect of Example 1]
As described above, according to the first embodiment, a predetermined inquiry by a specific conversation person is performed using the reference person response information in which the content of the reference person responding to a predetermined inquiry item by the specific conversation person is recorded. A response information evaluation program for causing a computer to execute a method for evaluating simulated evaluator response information in which the evaluator's simulated response to an item is recorded, and a specific dialogue included in the reference user response information The correction value for correcting the talker's feature value of the participant is calculated from the deviation from the average value of the talker feature value of the other talker related to the predetermined inquiry item. Since the degree of coincidence with the simulated response information is calculated, the degree of coincidence is corrected by the correction value, and the evaluation result of the evaluator simulated response information related to the predetermined inquiry item is calculated. When evaluating the simulated response information of the evaluator who recorded the simulated response by the operator to be evaluated using the response information), a specific customer can be identified even under conditions where the number of response information that can be accumulated is small. As a result, the accuracy of evaluating the skill of the operator can be improved.

  In addition, according to the first embodiment, since the correction value is calculated for each work phase of the reception, the feature amount of a specific customer is corrected for each work phase of the reception even under a condition where the number of pieces of reception information that can be accumulated is small. As a result, it becomes possible to further improve the accuracy of evaluating the skill of the operator.

  In addition, according to the first embodiment, the correction value is calculated using one or more of the speech time obtained from the voice data or the information relating to the ease of hearing of the voice as the talker feature amount. Even when the number of response information that can be accumulated is small, the feature amount of a specific customer is corrected, so that the accuracy of evaluating the skill of the operator can be improved.

[Outline and Features of Response Information Evaluation Apparatus According to Embodiment 2]
Subsequently, the outline and characteristics of the reception information evaluation apparatus according to the second embodiment will be described with reference to FIGS. 36 and 37. FIG. 36 and FIG. 37 are diagrams for explaining the outline and features of the reception information evaluation apparatus according to the second embodiment.

  As shown in FIG. 36, the response information evaluation apparatus according to the second embodiment provides a reference person who is an exemplary operator for a plurality of inquiry items (for example, inquiry item Q10, inquiry item Q20, inquiry item Q100, etc.). Reference person reception information that records the contents of the reception is stored in advance. Further, as shown in FIG. 36, the reception information evaluation apparatus stores in advance reception person reception information in which the contents corresponding to a plurality of reception persons are recorded using the reference person reception information. Further, as shown in FIG. 37, the response information evaluation apparatus records the contents actually received by the evaluated person for a predetermined inquiry item (inquiry item Q100) from a specific dialogue person (customer X). Evaluator performance response information is stored in advance. In the second embodiment, conversation voice data, reception record, reception time information, and memo data are stored in advance as reception information. In FIG. 36 and FIG. 37, the dialog voice data and other reception records are stored. It is roughly divided into two.

  First, the response information evaluation apparatus according to the second embodiment determines the degree of coincidence between the response information and the reference person response information, and the time required to input the response information and the time required to input the reference information. An input time ratio is calculated for each question and for each responder, and a degree-of-match function for calculating the degree of match for each question is derived for each question using the input time ratio as a variable from the calculated degree of match and input time ratio.

  Next, the response information evaluation device calculates the interviewer characteristic amount of the customer X included in the evaluation target actual response information in which the actual response to the evaluation target is recorded for the predetermined inquiry item Q100 by the customer X. A correction value for correction is calculated from a deviation from the average value of the conversation person feature amount of another conversation person related to the predetermined inquiry item Q100 (see (1) in FIG. 37).

  Then, the response information evaluation apparatus includes the reference person response information related to the predetermined inquiry item Q100 by the customer Y having the same correction value as the correction value and the time related to the input of the evaluation subject actual response information for which the correction value is calculated. By substituting the input time ratio with the time related to the input into the derived matching degree function, the degree of matching between the evaluated person actual response information and the reference person response information is calculated (see (2) in FIG. 37). ).

  Subsequently, the response information evaluation apparatus according to the second embodiment corrects the calculated degree of coincidence with the calculated correction value, and calculates an evaluation result of the evaluated person actual response information related to the inquiry item Q100 (FIG. 37 (3)).

  For this reason, the response information evaluation apparatus according to the second embodiment determines the number of response information that can be accumulated when evaluating the evaluation subject actual response information in which the evaluation target operator actually records the response. Even under a small number of conditions, the feature amount of a specific customer is corrected. As a result, the accuracy of evaluating the skill of the operator can be improved.

[Configuration of Response Information Evaluation Apparatus According to Second Embodiment]
Next, the configuration of the reception information evaluation apparatus according to the second embodiment will be described with reference to FIGS. FIG. 38 is a block diagram illustrating the configuration of the reception information evaluation apparatus according to the second embodiment, and FIG. 39 is a diagram for explaining an output example of the evaluation result of the evaluated person performance reception information, and FIG. 43 is a diagram for explaining the coincidence degree function deriving unit, FIG. 44 is a diagram for explaining the correction value calculating unit, and FIGS. 45 and 46 are for explaining the coincidence degree calculating unit. 47 to 50 are diagrams for explaining the evaluation result calculation unit.

  As shown in FIG. 38, the reception information evaluation device 40 is mainly composed of an input unit 41, an output unit 42, an input / output control IF unit 43, a storage unit 50, and a control unit 60.

  The input unit 41 is an input unit that inputs data used for various types of processing by the control unit 60 and operation instructions for performing various types of processing using a microphone, a keyboard, a storage medium, or communication. For example, reference person response information that records the contents that the reference person responds to multiple inquiry items, response information that records the contents that the respondent responds to multiple inquiry items, and a specific interlocutor The evaluation subject actual response information in which the content of the evaluation subject actually responding to the predetermined inquiry item is input, or an operation instruction for evaluating the evaluation subject actual response information is input.

  As a specific example, when the reference person, the responder, and the person to be evaluated respond to the inquiry item by the conversation person in the reception information evaluation apparatus 40, as shown in FIG. In accordance with a screen output to the monitor by the output unit 42 to be described later, information such as the affiliation code, company name, organization name, telephone number, name, etc. of the conversation person, inquiry contents, correspondence contents, correspondence results, etc. are input. The reference person reception information input by the input unit 41 is stored in a reference person reception information storage unit 51 described later, and the reception person reception information input by the input unit 41 is the reception person reception information storage unit 52 described later. The person-to-be-evaluated reception information input by the input unit 41 is stored in the person-to-be-evaluated result response information storage unit 55 described later.

  The output unit 42 is an output unit that outputs the results of various processes by the control unit 60 and operation instructions for performing various processes to a monitor, a printer, a speaker, or the like. For example, the evaluator actually responds to a screen for allowing the input unit 41 to input response information that the reference person or the responder has responded to a plurality of inquiry items, or to a predetermined inquiry item by a specific dialogue person. The screen for allowing the input unit 41 to input the received response information, the evaluation result of the evaluation target actual response information in which the content of the evaluation target actually recorded, and the like are output.

  Specifically, as described above, when the reference person, the responder, and the person to be evaluated respond to the inquiry item by the conversation person in the reception information evaluation apparatus 40, as shown in FIG. The output unit 42 outputs a screen for allowing the input unit 41 to input information such as the affiliation code, company name, organization name, telephone number, name, etc., inquiry contents, correspondence contents, correspondence results, etc. of the conversation person. As shown in FIG. 4, the output unit 42 outputs a screen for input by the input unit 41 for each inquiry content (for example, question content, complaint content, etc.) from the interlocutor. Also, as shown in FIG. 39, the evaluation result of the evaluated person performance response information is output in association with the evaluated person name (“evaluated person_a” and the evaluation result (“28”)).

  The input / output control IF unit 43 is means for controlling data transfer between the input unit 41 and the output unit 42, and the storage unit 50 and the control unit 60.

  The storage unit 50 is a storage unit that stores data used for various types of processing by the control unit 60, and particularly as closely related to the present invention, as shown in FIG. 38, a reference person reception information storage unit 51, Agent response information storage unit 52, inquiry item specific response information storage unit 53, matching degree function storage unit 54, evaluated person performance response information storage unit 55, correction value storage unit 56, and matching degree storage unit 57 And an evaluation result storage unit 58.

  Among the storage units 50, the reference person reception information storage unit 51 is information required for a plurality of respondents to respond to a plurality of inquiry items in the reception information evaluation apparatus 40, and includes an input unit 41 is a storage means for storing the reference person response information input by 41. Specifically, dialogue voice data, response record, response time information, memo data, and the like, which are reference person response information in which the reference person has responded to a plurality of inquiry items, are stored. For example, as shown in FIG. 6, the reference person reception information 51 stores information such as the affiliation code, company name, organization name, telephone number, and name of the conversation person, inquiry contents, and response contents as reception records. For example, as illustrated in FIG. 7, the reference person reception information storage unit 51 stores memo data that is a memo written as a material for which a reference record is created by the reference person.

  The response information storage unit 52 is a storage unit that stores information on response information input by the input unit 41, which is information that a plurality of respondents have responded to a plurality of inquiry items. Specifically, using the reference person response information that records the contents that the reference person has responded to a plurality of inquiry items, the dialogue voice data that is the response information of the person who has recorded the contents that each of the responders has recorded, Records, response time information, memo data, etc. are stored.

  The inquiry item-specific response information storage unit 53 includes response information stored in a reference person response information storage unit 51, an addressee response information storage unit 52, and an evaluated person performance response information storage unit 55 described later. It is a storage means for storing information to which information is assigned by the reproduction logging unit 61. Specifically, an operation log and operation time information are given by a response information reproduction logging unit 61 described later, and the response information classified for each inquiry item is stored.

  For example, in the reference person reception information storage unit 51, dialogue voice data between the “customer A” who is a dialogue person and “reference person_a” which is a reference person is stored as reference person reception information. As illustrated in FIG. 9, the storage unit 53 stores the conversation voice data acquired by the customer information A and the reference person_a as separate conversation voice data by the reception information reproduction logging unit 61. Further, for example, the reference person reception information storage unit 51 stores the memo data shown in FIG. 7 as reference person reception information, but the inquiry item-specific reception information storage unit 53 stores the reference person as shown in FIG. Is stored in the reception information reproduction logging unit 61 in which the order of information and time information are associated.

  For example, the reference person reception information storage unit 51 stores the reception record shown in FIG. 6 as reference person reception information. The inquiry item-specific reception information storage unit 53 stores the reception information as shown in FIG. The reproduction logging unit 61 associates the operation contents, operation time, and dialogue voice data, and further, a response number (for example, “A0001”) to be managed by the inquiry item-specific response information storage unit 53 is given. The information to which the performance flag indicating that it is the reception information in which the content of the reference person actually responding to the inquiry item is recorded is stored. Further, for example, the inquiry item-specific response information storage unit 53 stores the response information shown in FIG. 11 categorized by inquiry item (for example, “[question] net connection”) as shown in FIG. . As shown in FIG. 13, the inquiry item response information storage unit 53 stores various types of response information stored in association with the response number (eg, “A0001”) as a key.

  The coincidence degree function storage unit 54 is a storage unit that stores the coincidence degree function derived by the coincidence degree function deriving unit 62 described later. Specifically, a coincidence function for calculating the coincidence with the input time ratio as a variable is stored.

  The evaluated person performance response information storage unit 55 is storage information that is evaluated by the response information evaluation apparatus 40 and is storage means that stores the evaluated person result response information input by the input unit 41. Specifically, dialog voice data, response record, response time information, memo data, which is the actual response information of the evaluator, which records the actual response of the evaluator to a predetermined inquiry item by a specific interlocutor Memorize etc.

  The correction value storage unit 56 is a storage unit that stores a correction value calculated by a correction value calculation unit 63 described later. More specifically, a correction value for correcting the interlocutor feature amount of a specific interlocutor included in the evaluated person performance response information is stored. For example, as illustrated in FIG. 16, the correction value storage unit 56 includes an average value of the conversation person feature amount (“average conversation person feature amount”) and a customer conversation person feature amount (“interaction person feature amount”). And the customer's correction value ("correction value"). In the second embodiment, it is assumed that the inquiry is composed of the four phases of “reception phase”, “inquiry content grasping phase”, “problem factor identification phase”, and “problem solving phase” as the response work phase. Therefore, the correction value is also stored for each of these work phases.

  The coincidence degree storage unit 57 is a storage unit that stores the coincidence degree calculated by the coincidence degree calculation unit 64 described later. Specifically, the degree of coincidence between the person-to-be-evaluated actual reception information and the reference person reception information calculated by substituting into the coincidence degree function is stored.

  The evaluation result storage unit 58 is a storage unit that stores an evaluation result calculated by an evaluation result calculation unit 65 described later. Specifically, the evaluation result of the person-to-be-evaluated performance response information related to a predetermined inquiry item in which the degree of coincidence calculated by the degree of coincidence calculation unit 64 is corrected by the correction value calculated by the correction value calculation unit 63 is stored. . For example, as shown in FIG. 39, the evaluated person name (“evaluated person name”) and the evaluation result of the evaluated person achievement response information (“evaluated person achievement skill”) are stored in association with each other.

  Here, returning to FIG. 38, the control unit 60 is a control means for controlling the reception information evaluation apparatus 40 to execute various processes, and particularly as closely related to the present invention, as shown in FIG. In addition, a response information reproduction logging unit 61, a matching degree function deriving unit 62, a correction value calculating unit 63, a matching degree calculating unit 64, and an evaluation result calculating unit 65 are provided.

  In the control unit 60, the reception information reproduction logging unit 61 includes the response information stored in the reference person reception information storage unit 51, the reception person reception information storage unit 52, and the evaluated person performance reception information storage unit 55. It is a means for assigning and classifying information. Specifically, an operation log and operation time information are added to the inquiry information stored in the reference person reception information storage unit 51, the reception person reception information storage unit 52, and the evaluated person actual reception information storage unit 55, and the inquiry is made. Each item is classified and stored in the inquiry item-specific response information storage unit 53.

  The coincidence degree function deriving unit 62 is a means for deriving a coincidence degree function for calculating the coincidence degree using the input time ratio as a variable. Specifically, the degree of coincidence between the response information and the reference person response information stored in the inquiry item-specific response information storage unit 53, and the time required to input the response information and the reference person response information are input. The input time ratio with respect to the time is calculated for each question and for each respondent, and a matching degree function for each question is derived from the calculated matching degree and input time ratio, and stored in the matching degree function storage unit 54. The degree-of-match function deriving unit 62 corresponds to the “match-degree function deriving procedure” described in the claims.

  Here, the response information evaluation apparatus 10 according to the second embodiment calculates a coincidence degree function for each question. For example, as shown in FIG. 40, the reference person response information in which the reference person responds to the question q1 is recorded. , The input time ratio between the time related to the input of the responder information by the responder_a and the time related to the input of the reference person response information, and the response information and the reference of the respondent who recorded the contents responded by the responder_a A plot of the degree of coincidence with the person's reception information is indicated by (1.03, 0.99). Using the reference person's reception information that records the contents of the reference person's response to the question q1, The input time ratio between the time related to the input of the response information by the responder_b and the time related to the input of the response information of the reference person, and the response information of the response person and the reference person response information in which the content of the response of the response person_b is recorded A plot of the degree of match for ( It is shown in .16,0.60). Similarly, by plotting the degree of coincidence and the input time ratio for a plurality of respondents, the degree of coincidence function relating to the question q1 is derived and represented by the mathematical formula shown in FIG. 42 and 43 exemplify the degree of coincidence function derivation based on specific data.

  The correction value calculation unit 63 is a means for calculating a correction value for correcting the feature amount of a specific talker. Specifically, a correction value for correcting the dialogue feature amount of a specific dialogue person included in the person-to-be-evaluated performance reception information stored in the inquiry item-specific response information storage unit 53 is set as a predetermined inquiry item. The correction value is calculated from the deviation from the average value of the other talker's feature value of the talker and is stored in the correction value storage unit 56. The correction value calculation unit 63 corresponds to the “correction value calculation procedure” recited in the claims.

  Here, the response information evaluation apparatus 10 according to the second embodiment is configured to process various response information in response work phases (for example, “reception phase”, “inquiry content grasping phase”, “problem factor identification phase”, “problem solving phase”). Etc.) and evaluate each. For this reason, the correction value calculation unit 63 according to the second embodiment uses the dialog voice data, the response time information, the operation log, and the operation time information among the evaluation target actual response information that the evaluated person has responded to. The correspondence between the voice of the data and the work phase is estimated, and the utterance time of the talker for each work phase is calculated.

  Next, the correction value calculation unit 63 totals the customer utterance time for each of the “reception phase”, “inquiry content grasping phase”, “problem factor identification phase”, and “problem solving phase”, and these are summed for each work phase. It is assumed that Then, the correction value calculation unit 63 calculates the difference between the interlocutor feature quantity of the specific interlocutor related to the predetermined inquiry item and the average value of the interlocutor feature quantities of the other interlocutors, and a Gaussian error function As shown in FIG. 44, a correction value for correcting the interlocutor feature amount of a specific interlocutor included in the evaluated person performance response information is calculated and stored in the correction value storage unit 56.

  The coincidence degree calculation unit 64 is a means for calculating the coincidence degree between the evaluated person performance reception information and the reference reception information. Specifically, it is necessary to input the reference person's response information related to a predetermined inquiry item by a dialogue person having the same correction value as this correction value and the time related to the input of the evaluation subject actual response information for which the correction value is calculated. The coincidence degree is calculated by substituting the input time ratio with the time into the coincidence degree function derived by the coincidence degree function deriving unit 62, and is stored in the coincidence degree storage unit 57. The coincidence calculation unit 64 corresponds to the “coincidence calculation procedure” described in the claims.

  Here, in the coincidence degree calculation unit 64, a predetermined inquiry item by a conversation person having the same correction value as the correction value for correcting the conversation person feature amount of a specific conversation person included in the evaluated person performance response information. It is necessary to use the reference person handling information related to. Specifically, in FIG. 45, the person to be evaluated_a actually responds to the question q1 by the customer B (response number: AXXY7). Further, the time required for inputting the evaluation subject actual response information in which the response corresponding to the response number AXXY7 is recorded is ta_q1, and the correction value for correcting the conversation person feature amount of the customer B is U. Here, in the coincidence calculation unit 64, it is necessary to use the reference person reception information related to the question q1 by the conversation person whose correction value is U for the coincidence calculation. Then, in FIG. 45, the response number AXXY9 that the reference person_k is responding to the question q1 by the customer C is the reference person reception information that is the same inquiry item and has the same correction value. Therefore, the coincidence degree calculation unit 64 obtains an input time ratio between the time ta_q1 related to the input of the person-to-be-evaluated actual response information of the response number AXXY7 and the time tk_q1 related to the input of the reference person response information of the response number AXXY9. Assign to the degree function. For example, as shown in FIG. 46, since the coincidence degree function is derived for each question by the coincidence degree function deriving unit 62, by substituting the input time ratio corresponding to the question number into the coincidence degree function, The degree of coincidence of is calculated.

  The evaluation result calculation unit 65 is a means for calculating the evaluation result of the evaluated person performance response information. Specifically, using the correction value calculated by the correction value calculation unit 63 and the degree of coincidence calculated by the degree of coincidence calculation unit 64, the evaluation result of the person-to-be-evaluated actual response information related to a predetermined inquiry item is obtained. The calculation result is stored in the evaluation result storage unit 58. The evaluation result calculation unit 65 corresponds to the “evaluation result calculation procedure” described in the claims.

  Here, the evaluation result calculation part 65 in Example 2 calculates the evaluation result of the to-be-evaluated person response information by the mathematical formula shown in FIG. First, the left side Y of the expression (2-1) in FIG. 47 represents the evaluation result of the person-to-be-evaluated actual response information, and the right side U (i) is the feature amount of the talker in the work phase to which the question p (i) belongs. V represents a perfect score, and nU (i) represents the total score of the reference person's points. Further, the right side F (i) is a coincidence function, and the input time ratio between the time T (k_pi_0) related to the input of the reference person response information and the time T (j_pi_1) related to the input of the evaluated person actual response information is substituted. Is done.

  Next, Equations (2-2) to (2-6) in FIG. 48 represent the evaluation results of the person-to-be-evaluated performance response information represented by Equation (2-1) in FIG. 47 for each work phase. It is a thing. The expression (2-2) in FIG. 48 is an evaluation result of the “acceptance phase” and indicates that the question numbers i are 1 to 5. Also, the expression (2-3) in FIG. 48 is the evaluation result of the “inquiry content grasping phase” and indicates that the question number i is 6. The expression (2-4) in FIG. 3) This is a modification of the equation. Further, the expression (2-5) in FIG. 48 is an evaluation result of the “problem factor identification phase”, and indicates that the question number i is 7-9. Further, the expression (2-6) in FIG. 48 is an evaluation result of the “problem solving phase”, and indicates that the question number i is 10.

  Further, the expression (2-7) in FIG. 49 is an expression in which the right side nU (i) of the expression (2-1) in FIG. 47 is specifically developed, and the question number i of the “reception phase” is 1 to 5 Therefore, since the coefficient of the talker feature amount U (reception_AXXXY9) in the “reception phase” is 5 and the question number i in the “inquiry content grasping phase” is 6, the “inquiry content grasping phase” Since the coefficient of the talker feature amount U (Understanding_AXXXY9) is 1 and the question number i of the “problem factor identification phase” is 7 to 9, the talker feature amount U ( Since the coefficient of specific_AXXXY9) is 3 and the question number i of the “problem solving phase” is 10, the coefficient of the interlocutor feature amount U (solution_AXXXY9) in the “problem solving phase” is 1.

  50, the evaluation result Y (reception_AXXXY7) of the “reception phase”, the evaluation result Y (recognition_AXXXY7) of the “inquiry content grasping phase”, the “problem factor identification phase” "Evaluation result Y (specific_AXXXY7)" and "Problem solving phase" evaluation result Y (Solution_AXXXY7) multiplied by a normalization coefficient (In the formula (2-8), the maximum score V is 100. Is divided by the reciprocal of the normalization coefficient) is the evaluation result of the person-to-be-evaluated performance response information.

[Processing by Response Information Evaluation Apparatus in Example 2]
Next, the process by the reception information evaluation apparatus in Example 2 is demonstrated using FIGS. 51-53. 51 is a flowchart showing the procedure of the reception information evaluation process in the second embodiment, FIG. 52 is a flowchart showing the procedure of the matching degree function deriving process in the second embodiment, and FIG. 53 is a match in the second embodiment. It is a flowchart which shows the procedure of a degree calculation process.

[Response information evaluation process]
First, as shown in FIG. 51, the reception information evaluation apparatus 40 according to the second embodiment receives input of reference person reception information in which the reference person has received information on a predetermined inquiry item, and stores reference person reception information storage. The data is stored in the unit 51 (step S5101).

  Next, the reception information evaluation device 40 uses the reference person reception information in which the reference person's response contents are recorded in response to a predetermined inquiry item, and the reception person who records the contents that the plurality of respondents respectively simulatedly respond to. The input of the reception information is received and stored in the reception person reception information storage unit 52 (step S5102).

  Then, the reception information evaluation apparatus 40 adds an operation log and operation time information to the information stored in the reference person reception information storage unit 51 and the information stored in the reception person reception information storage unit 52, and corresponds to the inquiry item. The information is stored in the inquiry item-specific response information storage unit 53 (step S5103).

  Next, the response information evaluation device 40 inputs the degree of coincidence of each question between the customer response information and the reference customer response information stored in the response information storage unit 53 for each inquiry item, and the input of the customer response information for each question. The input time ratio between the time related to the input and the time related to the input for each question of the reference person response information is calculated for each responder, and the degree of coincidence is calculated using the input time ratio as a variable from the calculated coincidence and input time ratio. A matching degree function for each question is derived and stored in the matching degree function storage unit 54 (step S5104).

  Subsequently, the reception information evaluation device 40 receives an evaluation instruction of the evaluation target actual response information in which the evaluation target actually records the contents of the predetermined inquiry item by a specific dialogue person (Yes in Step S5105). ), An operation log and operation time information are added to the information stored in the to-be-evaluated person actual response information storage unit 55, classified by inquiry item, and stored in the inquiry item-specific response information storage unit 53 (step S5106). . If the evaluation instruction for the person-to-be-evaluated actual response information is not received (No in step S5105), the reception information evaluation apparatus 40 waits until the evaluation instruction is received.

  Then, the response information evaluation device 40 sets a correction value for correcting the interlocutor feature amount of a specific interlocutor included in the evaluator performance response information stored in the inquiry item-specific response information storage unit 53 to a predetermined value. The correction value is calculated from the deviation from the average value of the conversation person feature value of the other conversation person related to the inquiry item, and stored in the correction value storage unit 56 (step S5107).

  Next, the reception information evaluation apparatus 40 relates to a predetermined inquiry item by a conversation person having the same correction value and the time related to the input of the evaluation subject actual reception information stored in the inquiry information storage unit 53 by inquiry item. By substituting the input time ratio with the time related to the input of the reference person reception information into the coincidence degree function, the coincidence degree between the evaluated person actual reception information and the reference person reception information is calculated and stored in the coincidence degree storage unit 57 (Step S5108).

  Subsequently, the reception information evaluation apparatus 40 corrects the coincidence stored in the coincidence storage 57 with the correction value stored in the correction storage 56, and performs the evaluated person actual response related to the predetermined inquiry item. An evaluation result of information is calculated and stored in the evaluation result storage unit 58 (step S5109).

[Matching function derivation process]
Here, the coincidence function derivation process (step S5104 in FIG. 51) in the second embodiment will be specifically described with reference to FIG. 52. In the response information evaluation apparatus 40, the reference person responds to a plurality of inquiry items. A test to be performed using the reference person reception information in which the contents are recorded is performed on other responders (step S5201).

  Next, the matching degree function deriving unit 62 receives an input of a set of matching degree and input time ratio of the contents of the response record for each question (step S5202).

  Subsequently, the degree-of-match function deriving unit 62 plots a set of the degree of coincidence and the input time ratio taken into the Cartesian space having the input time ratio on the x-axis and the degree of coincidence on the y-axis (step S5203).

  The coincidence degree function deriving unit 62 performs a regression analysis on the plotted coincidence degree and the input time ratio to derive a coincidence degree function (step S5204).

  Subsequently, the matching degree function deriving unit 62 flags the question from which the matching degree function has been derived (step S5205).

  Next, the coincidence degree function deriving unit 62 determines whether there is a question without a flag (step S5206). If there is a question without a flag (Yes in step S5206), again, The input of the set of coincidence of the contents of the response record for each question and the input time ratio is received (step S5202). If there is no question without a flag (No at step S5206), the coincidence degree function deriving unit 62 ends the coincidence degree function deriving process.

[Matching degree calculation process]
Next, with reference to FIG. 53, the degree of coincidence calculation process (step S5108 in FIG. 51) in the second embodiment will be described in detail. Accepting an input of a correction value for correcting the interlocutor feature amount of a specific interlocutor included in the evaluator actual response information in which the evaluator actually responds to a predetermined inquiry item (step S5301).

  Then, the coincidence degree calculation unit 64 receives input of predetermined inquiry item information (step S5302).

  Subsequently, the degree of coincidence calculation unit 64 calculates a correction value of the reference person reception information when the reference person has handled the customer in the past (step S5303).

  Then, it is determined whether or not the inquiry items are the same (step S5304). If they are the same (Yes in step S5304), it is next determined whether or not the correction values are the same (step S5305). . If the inquiry items are not the same (No at step S5304), the process of receiving the input of the inquiry item is performed again.

  Next, when the correction value is the same value (Yes at Step S5305), the coincidence degree calculation unit 64 calculates the input time ratio for each question using the input time of the evaluated person performance response information (Step S5306). . If the correction value is not the same value (No at step S5305), the process for receiving the input of the inquiry item is performed again.

  Subsequently, the degree of coincidence calculation unit 64 gives the degree of coincidence for each question by giving the calculated input time ratio to the derived degree of coincidence function (step S5307).

  Then, when the coincidence calculation unit 64 obtains the coincidence of all questions (Yes at Step S5308), the coincidence degree calculation process is terminated. When the coincidence is not obtained (No at Step S5308), the evaluation subject performance response is performed again. The input time ratio for each question is calculated using the information input time.

[Effect of Example 2]
As described above, according to the second embodiment, there is provided a computer with a method for evaluating the evaluated person performance response information in which the evaluated person actually records the contents of the predetermined inquiry item by a specific dialogue person. A response information evaluation program to be executed, which includes response information recorded by each of the plurality of respondents using the reference person response information recorded by the reference person in response to a plurality of inquiry items. The degree of coincidence with the reference person response information, and the input time ratio between the time required for inputting the person contact information and the time relating to the input of the reference person response information are calculated for each of the respondents. Derivation of the degree of coincidence function that calculates the degree of coincidence from the input time ratio by using the input time ratio as a variable, and correction for correcting the interlocutor feature amount of a specific interlocutor included in the evaluated person performance response information Is calculated from a deviation from the average value of the other party's dialogue feature amount related to the predetermined inquiry item, and the correction value is calculated for the same time as the input of the evaluated person performance response information. By substituting the input time ratio with the time related to the input of the reference person response information related to the predetermined inquiry item by the interlocutor having the correction value into the derived matching degree function, the evaluated person actual response information and the reference person response The degree of coincidence with the information is calculated, the degree of coincidence is corrected by the correction value, and the evaluation result of the evaluated person performance response information related to the predetermined inquiry item is calculated. As a result of correcting the feature quantity of a specific customer even when the number of response information that can be accumulated is small, the accuracy of evaluating operator skills is improved. It becomes possible.

  Further, according to the second embodiment, a condition designation for selecting the evaluated person performance response information to be evaluated is received, and a specific dialogue person included in the evaluated person performance response information selected by the accepted condition specification is received. Since the correction value for correcting the feature value of the interlocutor is calculated, the number of response information that can be accumulated is small when evaluating the evaluated response performance information recorded by the operator to be evaluated. Even under conditions, it is possible to select the evaluation subject performance response information to be evaluated by specifying the conditions. As a result, it is possible to further improve the accuracy of evaluating the skill of the operator.

  In addition, according to the second embodiment, since the correction value is calculated for each work phase of reception, the feature amount of a specific customer is corrected for each work phase of the reception even under a condition where the number of response information that can be accumulated is small. As a result, it becomes possible to further improve the accuracy of evaluating the skill of the operator.

  In addition, according to the second embodiment, the correction value is calculated using one or more of the speech time obtained from the voice data or the information related to the ease of hearing of the voice as the talker feature amount. Even when the number of response information that can be accumulated is small, the feature amount of a specific customer is corrected, so that the accuracy of evaluating the skill of the operator can be improved.

[Other embodiments]
By the way, although the reception information evaluation apparatus which concerns on Example 1 and Example 2 was demonstrated so far, this invention may be implemented with another form other than an above-described Example. Therefore, in the following, another embodiment will be described as a reception information evaluation apparatus according to the third embodiment.

  In the first embodiment and the second embodiment described above, the correction value is calculated using the utterance time as the talker feature amount. However, the present invention is not limited to this, and the utterance time, the number of words, Alternatively, any method may be used as long as it indicates the feature of the talker by a quantity, such as calculating a correction value using any one or a plurality of pieces of information relating to the ease of hearing.

  In the first embodiment and the second embodiment, the case where the Gaussian error calculation function is used for calculating the correction value has been described. However, the present invention is not limited to this, and the deviation from the average value is calculated. Any method may be used as long as it is a correction method.

  In the first and second embodiments, the case where morpheme analysis is used for calculating the degree of coincidence has been described. However, the present invention is not limited to this, and is a technique that can measure the degree of coincidence of reception information. Any one is acceptable.

  In the above-described response information evaluation apparatus according to the second embodiment, the case where the matching degree function derivation process is performed before receiving the evaluation instruction of the evaluated person actual response information has been described. However, the present invention is not necessarily limited thereto. In the case where a matching degree function is derived in advance from the simulated person-to-be-evaluated information collected by the receiving information evaluation apparatus according to the first embodiment, or the matching information is collected by collecting the handling information as needed. As long as the coincidence function is derived before the process of calculating the coincidence in the evaluation of the evaluated person performance response information, such as when a function is derived, any may be used.

[Computer (Example 1)]
By the way, the various processes described in the first embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. Therefore, in the following, an example of a computer that executes a reception information evaluation program having the same function as in the first embodiment will be described with reference to FIG. FIG. 54 is a diagram illustrating the computer that executes the reception information evaluation program according to the first embodiment.

  As shown in FIG. 54, the reception information evaluation program (computer) 70 is configured by connecting a cache 71, a RAM 72, an HDD 73, a ROM 74, and a CPU 75 via a bus 76. Here, in the ROM 74, a reception information evaluation program that exhibits the same function as in the first embodiment, that is, as shown in FIG. 54, a reception information reproduction logging program 74a, a correction value calculation program 74b, and a coincidence degree calculation program. 74c and an evaluation result calculation program 74d are stored in advance.

  Then, the CPU 75 reads out and executes these programs 74a, 74b, 74c, and 74d, so that each program 74a, 74b, 74c, and 74d becomes the reception information reproduction logging process 75a, A correction value calculation process 75b, a coincidence degree calculation process 75c, and an evaluation result calculation process 75d. Each process 75a, 75b, 75c, and 75d corresponds to the reception information reproduction logging unit 31, the correction value calculation unit 32, the coincidence degree calculation unit 33, and the evaluation result calculation unit 34 illustrated in FIG.

  By the way, the above-described programs 74a, 74b, 74c, and 74d are not necessarily stored in the ROM 74. For example, a flexible disk (FD), a CD-ROM, an MO disk, and a DVD that are inserted into the computer 70. “Portable physical medium” such as a disk, magneto-optical disk, IC card, or “fixed physical medium” such as a hard disk drive (HDD) provided inside or outside the computer 70, and further, a public line, the Internet, It may be stored in “another computer (or server)” connected to the computer 70 via a LAN, WAN, etc., and the computer 70 may read out and execute the program from these.

[Computer (Example 2)]
The various processes described in the second embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. Therefore, in the following, an example of a computer that executes a reception information evaluation program having the same function as in the second embodiment will be described with reference to FIG. FIG. 55 is a diagram illustrating a computer that executes a reception information evaluation program according to the second embodiment.

  As shown in FIG. 55, the reception information evaluation program (computer) 80 is configured by connecting a cache 81, a RAM 82, an HDD 83, a ROM 84, and a CPU 85 via a bus 86. Here, in the ROM 84, a reception information evaluation program that exhibits the same function as in the second embodiment, that is, as shown in FIG. 55, a reception information reproduction logging program 84a, a matching degree function derivation program 84b, and a correction value calculation A program 84c, a matching degree calculation program 84d, and an evaluation result calculation program 84e are stored in advance.

  Then, the CPU 85 reads and executes these programs 84a, 84b, 84c, 84d, and 84e, so that each program 84a, 84b, 84c, 84d, and 84e reproduces the reception information as shown in FIG. A logging process 85a, a matching degree function deriving process 85b, a correction value calculating process 85c, a matching degree calculating process 85d, and an evaluation result calculating process 85e. Each process 85a, 85b, 85c, 85d, and 85e includes the reception information reproduction logging unit 61, the matching degree function deriving unit 62, the correction value calculating unit 63, the matching degree calculating unit 64, and the evaluation shown in FIG. Each corresponds to the result calculation unit 65.

  By the way, the above-mentioned programs 84a, 84b, 84c, 84d, and 84e are not necessarily stored in the ROM 84. For example, a flexible disk (FD), a CD-ROM, or an MO disk that is inserted into the computer 80. , “Portable physical media” such as DVD disks, magneto-optical disks, IC cards, etc., or “fixed physical media” such as hard disk drives (HDDs) provided inside and outside the computer 80, The program may be stored in “another computer (or server)” connected to the computer 80 via the Internet, LAN, WAN, etc., and the computer 80 may read and execute the program from these.

[System configuration, etc.]
In addition, among the processes described in the above embodiments, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed All or a part of the above can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Further, all or any part of each processing function performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

(Supplementary note 1) Using the reference person response information in which the reference person responds to a predetermined inquiry item by a specific dialogue person, the person to be evaluated for the predetermined inquiry item by the specific dialogue person Is a response information evaluation program for causing a computer to execute a method for evaluating simulated response information of a person to be evaluated, which records the content of response in a simulated manner,
The correction value for correcting the interlocutor feature amount of the specific interlocutor included in the reference person response information is an average value of interlocutor feature amounts of other interrogators related to the predetermined inquiry item. A correction value calculation procedure to calculate from the deviation;
A degree of coincidence calculation procedure for calculating a degree of coincidence between the reference person reception information and the evaluator simulated reception information;
The degree of coincidence calculated by the degree of coincidence calculation procedure is corrected by the correction value calculated by the correction value calculation step, and the evaluation result of the evaluator simulated reception information related to the predetermined inquiry item is calculated. Evaluation result calculation procedure to
A response information evaluation program for causing a computer to execute.

(Supplementary Note 2) A response information evaluation program for causing a computer to execute a method for evaluating an evaluated person actual response information in which a content of an evaluated person actually responding to a predetermined inquiry item by a specific dialogue person is recorded. ,
The degree of coincidence between the contact information and the reference information, which is recorded by the reference information recorded by the reference information recorded by the reference person in response to a plurality of inquiry items, In addition, an input time ratio between the time for inputting the response information of the responder and the time for input of the reference person response information is calculated for each response person, and the input time ratio is calculated from the calculated matching degree and input time ratio. A degree-of-match function derivation procedure for deriving a degree-of-match function that calculates the degree of match as a variable;
The correction value for correcting the interlocutor feature amount of the specific interlocutor included in the evaluated person performance response information is the average value of the interlocutor feature amount of the other interrogator related to the predetermined inquiry item. Correction value calculation procedure to calculate from the deviation of
Reference person response information related to the predetermined inquiry item by the interlocutor who has the same correction value as the time to input the evaluation subject actual response information whose correction value is calculated in the correction value calculation procedure By substituting the input time ratio with the time related to the input into the coincidence degree function derived by the coincidence degree function deriving procedure, the degree of coincidence between the evaluated person actual reception information and the reference person reception information is calculated. The matching score calculation procedure,
The degree of coincidence calculated by the degree of coincidence calculation procedure is corrected by the correction value calculated by the correction value calculation step, and the evaluation result of the evaluated person performance response information related to the predetermined inquiry item is calculated. Evaluation result calculation procedure to
A response information evaluation program for causing a computer to execute.

(Supplementary Note 3) A condition designation acceptance procedure for accepting a condition designation for selecting evaluation subject performance response information to be evaluated is further provided.
The correction value calculation procedure includes:
Supplementary note 2 for calculating a correction value for correcting a feature value of a specific conversation person included in the evaluated person performance response information selected by the condition designation accepted by the condition designation acceptance procedure. Response information evaluation program described in 1.

(Additional remark 4) The said correction value calculation procedure calculates the said correction value for every work phase of reception, The reception information evaluation program as described in any one of Additional remarks 1-3 characterized by the above-mentioned.

(Additional remark 5) The said correction value calculation procedure uses the said one or more as information about the speech time obtained from audio | voice data, or the information which concerns on an audio | voice intelligibility as said dialogue person feature-value, and calculates the said correction value. The reception information evaluation program according to any one of appendices 1 to 4, wherein the reception information evaluation program is calculated.

(Supplementary note 6) Evaluated person for the predetermined inquiry item by the specific dialogue person using the reference person response information in which the reference person responds to the predetermined inquiry item by the specific dialogue person. Is a response information evaluation device that evaluates the simulated response information of the person to be evaluated, which records the content of the response in a simulated manner,
The correction value for correcting the interlocutor feature amount of the specific interlocutor included in the reference person response information is an average value of interlocutor feature amounts of other interrogators related to the predetermined inquiry item. Correction value calculating means for calculating from the deviation;
A degree of coincidence calculating means for calculating a degree of coincidence between the reference person reception information and the evaluated person simulated reception information;
The degree of coincidence calculated by the degree of coincidence calculating unit is corrected by the correction value calculated by the correction value calculating unit, and the evaluation result of the evaluator simulated reception information related to the predetermined inquiry item is calculated. An evaluation result calculating means for
A reception information evaluation apparatus characterized by comprising:

(Supplementary note 7) A response information evaluation apparatus for evaluating the evaluated person actual response information in which the evaluated person actually records the contents of the predetermined inquiry by a specific dialogue person,
The degree of coincidence between the response information of the responder and the reference information of the reference, in which the information of the responses of the plurality of respondents is recorded using the reference information of the reference that records the response of the reference to the multiple inquiry items, In addition, an input time ratio between the time for inputting the response information of the responder and the time for input of the reference person response information is calculated for each response person, and the input time ratio is calculated from the calculated matching degree and input time ratio. A degree-of-match function derivation means for deriving a degree-of-match function that calculates the degree of match as a variable;
The correction value for correcting the interlocutor feature amount of the specific interlocutor included in the evaluated person performance response information is the average value of the interlocutor feature amount of the other interrogator related to the predetermined inquiry item. Correction value calculation means for calculating from the deviation of
The reference person response information related to the predetermined inquiry item by the interlocutor having the same correction value as the time for the input of the evaluated person performance response information for which the correction value is calculated by the correction value calculation means By substituting the input time ratio with the time related to the input into the coincidence degree function derived by the coincidence degree function deriving means, the degree of coincidence between the evaluated person actual reception information and the reference person reception information is calculated. A degree-of-match calculation means;
The degree of coincidence calculated by the degree of coincidence calculating unit is corrected by the correction value calculated by the correction value calculating unit, and the evaluation result of the evaluated person performance response information related to the predetermined inquiry item is calculated. An evaluation result calculating means for
A reception information evaluation apparatus characterized by comprising:

(Supplementary note 8) Using the reference person response information in which the reference person responds to a predetermined inquiry item by a specific dialogue person, the person to be evaluated for the predetermined inquiry item by the specific dialogue person Is a response information evaluation method for evaluating the simulated response information of the evaluator who recorded the simulated response.
The correction value for correcting the interlocutor feature amount of the specific interlocutor included in the reference person response information is an average value of interlocutor feature amounts of other interrogators related to the predetermined inquiry item. A correction value calculating step to calculate from the deviation;
A degree of coincidence calculation step for calculating a degree of coincidence between the reference person reception information and the evaluator simulated reception information;
The degree of coincidence calculated by the degree of coincidence calculation step is corrected by the correction value calculated by the correction value calculation step, and the evaluation result of the evaluator simulated response information related to the predetermined inquiry item is calculated. An evaluation result calculation step to perform,
A response information evaluation method characterized by including

(Supplementary note 9) A response information evaluation method for evaluating an evaluated person actual response information in which a content of an evaluated person actually responding to a predetermined inquiry item by a specific dialogue person is recorded,
The degree of coincidence between the contact information and the reference information, which is recorded by the reference information recorded by the reference information recorded by the reference person in response to a plurality of inquiry items, In addition, an input time ratio between the time for inputting the response information of the responder and the time for input of the reference person response information is calculated for each response person, and the input time ratio is calculated from the calculated matching degree and input time ratio. A degree-of-match function derivation step for deriving a degree-of-match function that calculates the degree of match as a variable;
The correction value for correcting the interlocutor feature amount of the specific interlocutor included in the evaluated person performance response information is the average value of the interlocutor feature amount of the other interrogator related to the predetermined inquiry item. A correction value calculation step of calculating from the deviation of
Reference person response information related to the predetermined inquiry item by a dialoguer having the same correction value as the time for inputting the evaluation subject actual response information for which the correction value is calculated in the correction value calculation step By substituting the input time ratio with the time related to the input into the coincidence degree function derived by the coincidence degree function deriving step, the degree of coincidence between the evaluation subject actual reception information and the reference person reception information is calculated. A matching degree calculation step;
The degree of coincidence calculated by the degree of coincidence calculation step is corrected by the correction value calculated by the correction value calculation step, and the evaluation result of the evaluation subject actual response information related to the predetermined inquiry item is calculated. An evaluation result calculation step to perform,
A response information evaluation method characterized by including

  As described above, the response information evaluation program according to the present invention uses the reference person response information in which the reference person responds to a predetermined inquiry item by a specific dialog person, and uses the reference person response information. It is useful for evaluating the evaluator simulated response information in which the content of the evaluator responding to the predetermined inquiry item is simulated, especially under conditions where the number of response information that can be accumulated is small. It is suitable for improving the accuracy of evaluating operator skills.

It is a figure for demonstrating the outline | summary and the characteristic of the reception information evaluation apparatus which concern on Example 1. FIG. It is a block diagram which shows the structure of the reception information evaluation apparatus which concerns on Example 1. FIG. It is a figure for demonstrating the screen image which inputs reception information (response recording). It is a figure for demonstrating the screen image for every inquiry content. It is a figure for demonstrating the output example of the evaluation result of to-be-evaluated person simulated reception information. It is a figure for demonstrating a reference person reception information storage part. It is a figure for demonstrating a reference person reception information storage part. It is a figure for demonstrating to-be-evaluated person simulated reception information storage part. It is a figure for demonstrating the response information memory | storage part classified by inquiry item. It is a figure for demonstrating the response information memory | storage part classified by inquiry item. It is a figure for demonstrating the response information memory | storage part classified by inquiry item. It is a figure for demonstrating the response information memory | storage part classified by inquiry item. It is a figure for demonstrating the response information memory | storage part classified by inquiry item. It is a figure for demonstrating the response information memory | storage part classified by inquiry item. It is a figure for demonstrating the response information memory | storage part classified by inquiry item. It is a figure for demonstrating a correction value memory | storage part. It is a figure for demonstrating a coincidence degree memory | storage part. It is a figure for demonstrating a reception information reproduction | regeneration logging part. It is a figure for demonstrating a correction value calculation part. It is a figure for demonstrating a correction value calculation part. It is a figure for demonstrating a correction value calculation part. It is a figure for demonstrating a correction value calculation part. It is a figure for demonstrating a coincidence degree calculation part. It is a figure for demonstrating a coincidence degree calculation part. It is a figure for demonstrating an evaluation result calculation part. It is a figure for demonstrating an evaluation result calculation part. It is a figure for demonstrating an evaluation result calculation part. It is a figure for demonstrating an evaluation result calculation part. It is a figure for demonstrating the flow of the information which concerns on the reception information in Example 1. FIG. 6 is a flowchart illustrating a procedure of a reception information evaluation process in the first embodiment. 6 is a flowchart illustrating a procedure of a reception information reproduction logging process according to the first embodiment. 6 is a flowchart illustrating a procedure of a reception information reproduction logging process according to the first embodiment. 6 is a flowchart illustrating a procedure of correction value calculation processing according to the first embodiment. 6 is a flowchart illustrating a procedure of matching degree calculation processing according to the first exemplary embodiment. 6 is a flowchart illustrating a procedure of matching degree calculation processing according to the first exemplary embodiment. It is a figure for demonstrating the outline | summary and the characteristic of the reception information evaluation apparatus which concern on Example 2. FIG. It is a figure for demonstrating the outline | summary and the characteristic of the reception information evaluation apparatus which concern on Example 2. FIG. It is a block diagram which shows the structure of the reception information evaluation apparatus which concerns on Example 2. FIG. It is a figure for demonstrating the output example of the evaluation result of to-be-evaluated person performance response information. It is a figure for demonstrating a coincidence degree function derivation | leading-out part. It is a figure for demonstrating a coincidence degree function derivation | leading-out part. It is a figure for demonstrating a coincidence degree function derivation | leading-out part. It is a figure for demonstrating a coincidence degree function derivation | leading-out part. It is a figure for demonstrating a correction value calculation part. It is a figure for demonstrating a coincidence degree calculation part. It is a figure for demonstrating a coincidence degree calculation part. It is a figure for demonstrating an evaluation result calculation part. It is a figure for demonstrating an evaluation result calculation part. It is a figure for demonstrating an evaluation result calculation part. It is a figure for demonstrating an evaluation result calculation part. It is a flowchart which shows the procedure of the reception information evaluation process in Example 2. 12 is a flowchart illustrating a procedure of matching degree function derivation processing according to the second embodiment. 12 is a flowchart illustrating a procedure of matching degree calculation processing according to the second embodiment. It is a figure which shows the computer which performs the reception information evaluation program which concerns on Example 1. FIG. It is a figure which shows the computer which performs the reception information evaluation program which concerns on Example 2. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Reception information evaluation apparatus 11 Input part 12 Output part 13 Input / output control IF part 20 Storage part 21 Reference person reception information storage part 22 Evaluator simulated reception information storage part 23 Inquiry item specific reception information storage part 24 Correction value storage part 25 Matching degree storage section 26 Evaluation result storage section 30 Control section 31 Response information reproduction logging section 32 Correction value calculation section 33 Matching degree calculation section 34 Evaluation result calculation section 40 Response information evaluation apparatus 41 Input section 42 Output section 43 Input / output control IF section DESCRIPTION OF SYMBOLS 50 Storage part 51 Reference person reception information storage part 52 Response person response information storage part 53 Response information storage part according to inquiry item 54 Matching degree function storage part 55 Evaluated person result reception information storage part 56 Correction value storage part 57 Matching degree storage part 58 evaluation result storage unit 60 control unit 61 reception information reproduction logging unit 62 coincidence function derivation unit 63 correction value calculation unit 64 Matching degree calculation unit 65 Evaluation result calculation unit 70 Reception information evaluation program (computer)
71 Cache 72 RAM
73 HDD
74 ROM
75 CPU
76 Bus 80 Reception information evaluation program (computer)
81 Cache 82 RAM
83 HDD
84 ROM
85 CPU
86 Bus

Claims (5)

The evaluator simulates the predetermined inquiry item by the specific dialogue person using the reference person response information in which the reference person has responded to the predetermined inquiry item by the specific dialogue person. A response information evaluation program for causing a computer to execute a method for evaluating simulated evaluator response information in which response content is recorded,
The correction value for correcting the interlocutor feature amount of the specific interlocutor included in the reference person response information is an average value of interlocutor feature amounts of other interrogators related to the predetermined inquiry item. A correction value calculation procedure to calculate from the deviation;
A degree of coincidence calculation procedure for calculating a degree of coincidence between the reference person reception information and the evaluator simulated reception information;
The degree of coincidence calculated by the degree of coincidence calculation procedure is corrected by the correction value calculated by the correction value calculation step, and the evaluation result of the evaluator simulated reception information related to the predetermined inquiry item is calculated. Evaluation result calculation procedure to
A response information evaluation program for causing a computer to execute. A response information evaluation program for causing a computer to execute a method for evaluating an evaluated user performance response information in which a content of an evaluated user actually recorded in response to a predetermined inquiry item by a specific dialogue person,
The degree of coincidence between the contact information and the reference information, which is recorded by the reference information recorded by the reference information recorded by the reference person in response to a plurality of inquiry items, In addition, an input time ratio between the time for inputting the response information of the responder and the time for input of the reference person response information is calculated for each response person, and the input time ratio is calculated from the calculated matching degree and input time ratio. A degree-of-match function derivation procedure for deriving a degree-of-match function that calculates the degree of match as a variable;
The correction value for correcting the interlocutor feature amount of the specific interlocutor included in the evaluated person performance response information is the average value of the interlocutor feature amount of the other interrogator related to the predetermined inquiry item. Correction value calculation procedure to calculate from the deviation of
Reference person response information related to the predetermined inquiry item by the interlocutor who has the same correction value as the time to input the evaluation subject actual response information whose correction value is calculated in the correction value calculation procedure By substituting the input time ratio with the time related to the input into the coincidence degree function derived by the coincidence degree function deriving procedure, the degree of coincidence between the evaluated person actual reception information and the reference person reception information is calculated. The matching score calculation procedure,
The degree of coincidence calculated by the degree of coincidence calculation procedure is corrected by the correction value calculated by the correction value calculation step, and the evaluation result of the evaluated person performance response information related to the predetermined inquiry item is calculated. Evaluation result calculation procedure to
A response information evaluation program for causing a computer to execute. Further comprising a condition designation acceptance procedure for accepting a condition designation for selecting the evaluation subject performance response information to be evaluated,
The correction value calculation procedure includes:
The correction value for correcting the interlocutor feature quantity of a specific interlocutor included in the evaluated person performance response information selected by the condition designation accepted by the condition designation acceptance procedure is calculated. The reception information evaluation program described in 2.   The said correction value calculation procedure calculates the said correction value for every operation | work phase of reception, The reception information evaluation program as described in any one of Claims 1-3 characterized by the above-mentioned.   The correction value calculation procedure calculates the correction value by using one or more of the speech time obtained from voice data or information related to the ease of hearing of voice as the talker feature amount. The reception information evaluation program according to any one of claims 1 to 4, wherein

Images