KR20220075015A - Method for evaluating call counselor for inside sales and data analysis apparatus therefor - Google Patents

Abstract

Disclosed are a method for evaluating inside sales call agents and a data analysis apparatus therefor. The data analysis apparatus according to an embodiment includes an STT conversion unit that converts voice sales data generated through an inside sales call of an agent into text sales data, and a dictionary construction unit that builds a dictionary for information extraction from the converted text sales data. and an information extraction unit that extracts core information for call grade calculation from text sales data using the built-in dictionary, and a call grade calculator that calculates the grade for an inside sales call of an agent using the extracted core information; , and a call grade recommender that recommends the calculated call grade.

Description

Method for evaluating call counselor for inside sales and data analysis apparatus therefor

The present invention relates to a data analysis technology, and more particularly, to a technology for analyzing a sales call conversation based on voice recognition.

Voice recognition technology is naturally used in everyday life, such as car navigation, smartphone conversation systems, and interpretation and translation. In addition, voice recognition technology using neural networks such as deep learning is applied to call centers such as telecommunication companies, insurance companies, securities companies, and shopping malls to extend the scope of VOC (Voice of Customer) analysis, which was limited to text data, to the voice of customers. did.

According to an embodiment, an inside sales call agent evaluation method capable of systematic management and evaluation of inside sales call agents, and a data analysis apparatus for the same are proposed.

The data analysis apparatus according to an embodiment includes an STT conversion unit that converts voice sales data generated through an inside sales call of an agent into text sales data, and a dictionary construction unit that builds a dictionary for information extraction from the converted text sales data. and an information extraction unit that extracts core information for call grade calculation from text sales data using the built-in dictionary, and a call grade calculator that calculates the grade for an inside sales call of an agent using the extracted core information; , and a call grade recommender that recommends the calculated call grade.

The dictionary construction unit builds a dictionary of entity names corresponding to words belonging to a predetermined set among the customer's core vocabulary, builds a semantic dictionary corresponding to common nouns for understanding the meaning of sentences other than entity names, and asks the customer a question Construct a dialogue act dictionary corresponding to a sentence to determine whether the utterance is intended with any one of , answer, general, and solicitation, and a pattern dictionary corresponding to the result of synthesizing the entity name dictionary, semantic dictionary, and dialogue act dictionary can be constructed. .

The dictionary construction unit provides a representative keyword to the result of synthesizing the entity name dictionary, the semantic dictionary, and the dialogue act dictionary, and the representative keyword may be composed of keywords representing BANT information.

The information extraction unit may extract core information according to an index required for call grade calculation (hereinafter referred to as a 'call grade calculation index') with reference to the built-up information extraction dictionaries, and calculate the sound source playback time. The call rating calculation index may include at least one of a sales conversation time, a question score compared to a sales conversation time, a BANT weighted question score, a solicitation question score, and the number of keywords for each BANT.

The call grade calculation unit calculates the final call grade value through arithmetic operation of the weights for each indicator and the criteria for each project, based on the extracted core information, and the call grade recommender uses the final value calculated through the call grade calculation part to call According to the rating index, a call rating can be recommended. The criterion for each indicator is the differential classification of core information from each call grade calculation indicator, and the weight for each indicator indicates the importance of each call grade calculation indicator, and the value is distributed based on the sum of 1, and the call grade classification indicator is Differential call grades can be given to the final values obtained by arithmetic operation of the criteria for each indicator and the weights for each indicator.

The inside sales call agent evaluation method according to another embodiment includes the steps of, by a data analysis device, converting voice sales data generated through the agent's inside sales call into text sales data, and a dictionary for information extraction from the converted text sales data The step of constructing, the step of extracting core information for call grade calculation from the text sales data using the built dictionary, the step of calculating the grade for the inside sales call of the agent using the extracted core information, and calculation; and recommending an assigned call grade.

According to an inside sales call agent evaluation method and a data analysis apparatus therefor according to an embodiment, the sales call agent may be evaluated based on artificial intelligence with respect to the sales call conversation conducted by the agent. For example, by converting voice sales data, which is voice data, into text sales data through STT, and analyzing the text sales data according to an index required for call grade calculation, a sales call grade can be calculated and recommended.

Due to the development of artificial intelligence technology and the Corona era, call counseling is playing a key role in non-face-to-face channels. In this situation, the call consulting service is not only in resolving customer complaints simply, but is now also in charge of the call sales part that contributes to the increase in the company's sales. Systematic management and evaluation of call center agents is possible through the inside sales call agent evaluation method of the present invention and the data analysis device for the same. In order to give customers a positive service experience and lead to future profits, it is possible to provide an effective management strategy for call center agents.

It is expected that the inside sales call agent evaluation method and data analysis device for the same proposed in the present invention will have a good effect on call center management and strategy establishment in the post-corona era.

1 is a diagram showing the configuration of a data analysis apparatus according to an embodiment of the present invention;
2 is a view showing a detailed configuration of the control unit of FIG. 1 according to an embodiment of the present invention;
3 is a diagram illustrating a flow of a method for evaluating a call agent according to an embodiment of the present invention;
4 is a view showing an STT recognition process according to an embodiment of the present invention;
5 is a diagram illustrating an example of a dictionary for information extraction according to an embodiment of the present invention;
6 is a view showing an example of BANT information and representative keywords according to an embodiment of the present invention;
7 is a diagram illustrating an example of core information required for call class calculation according to an embodiment of the present invention;
8 is a diagram illustrating an example of a call grade recommendation result according to a call grade calculation according to an embodiment of the present invention.

Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

In describing the embodiments of the present invention, if it is determined that a detailed description of a well-known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted, and the terms to be described later are used in the embodiment of the present invention. As terms defined in consideration of the function of Therefore, the definition should be made based on the content throughout this specification.

Each block in the accompanying block diagram and combinations of steps in the flowchart may be executed by computer program instructions (execution engine), which are executed by the processor of a general-purpose computer, special-purpose computer, or other programmable data processing device. It may be mounted so that its instructions, which are executed by the processor of a computer or other programmable data processing device, create means for performing the functions described in each block of the block diagram or in each step of the flowchart.

These computer program instructions may also be stored in a computer-usable or computer-readable memory which may direct a computer or other programmable data processing device to implement a function in a particular manner, and thus the computer-usable or computer-readable memory. It is also possible to produce an article of manufacture containing instruction means for performing the functions described in each block of the block diagram or each step of the flowchart, the instructions stored in the block diagram.

And since the computer program instructions may be mounted on a computer or other programmable data processing device, a series of operational steps is performed on the computer or other programmable data processing device to create a computer-executable process to create a computer or other programmable data processing device. It is also possible that instructions for performing the data processing apparatus provide steps for executing functions described in each block in the block diagram and in each step in the flowchart.

Also, each block or step may represent a module, segment, or portion of code comprising one or more executable instructions for executing specified logical functions, and in some alternative embodiments the blocks or steps referred to in the block or steps. It should be noted that it is also possible for functions to occur out of sequence. For example, it is possible that two blocks or steps shown one after another may be performed substantially simultaneously, and also the blocks or steps may be performed in the reverse order of the corresponding functions, if necessary.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention illustrated below may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to more completely explain the present invention to those of ordinary skill in the art to which the present invention pertains.

1 is a diagram showing the configuration of a data analysis apparatus according to an embodiment of the present invention.

The data analysis device 1 may be operated by a business operator that provides a sales conversation analysis service for evaluating inside sales call agents or an entity under the supervision of the business operator. For example, the data analysis apparatus 1 may be operated by a marketing company or a business connection service provider, but the embodiment is not limited thereto. The data analysis device 1 may be a computing device capable of performing a predetermined calculation process and a communication process.

Inside sales refers to sales operations that can be performed in the office without customer visits. Compared to on-site sales, where a sales representative directly visits potential customers and conducts consultations, inside sales is different in that they communicate with others by phone, e-mail, or chatting tools.

Although inside sales is not yet popularized by many companies, it is changing the existing sales style, such as making good use of talent and preventing missed opportunities to foster potential customers due to the shortage of manpower. is becoming In addition, as un-tact becomes a reality due to the corona era, inside sales of non-face-to-face sales will become more important in the post-corona era. Accordingly, the importance of management and evaluation of call agents who identify potential customers is emerging. The present invention proposes a technique for managing and evaluating a call agent.

The development of sales conversation analysis technology helps a lot in the evaluation of inside sales call agents. When examining the evaluation of the existing call agents, they were evaluated using superficial indicators such as the number of calls processed per hour and the customer waiting time (seconds) before connecting to an agent, rather than evaluating the contents. However, in the present invention, the content of the sales conversation such as sales conversation time, question score versus sales conversation time, BANT weighted question score, solicitation question score, and the number of keywords for each BANT are analyzed and the agent's call can be evaluated using more in-depth indicators. suggest a way to

Referring to FIG. 1 , the data analysis apparatus 1 includes an input unit 10 , a control unit 12 , an output unit 14 , and a storage unit 16 .

The input unit 10 receives the sales data (hereinafter, referred to as 'voice sales data') in the form of voice exchanged with the customer by the inside call agent who needs analysis. In addition, it is possible to receive a manipulation signal by a user manipulation. Voice sales data is obtained from inside sales calls that occur in call centers such as telecommunication companies, insurance companies, securities companies, and shopping malls. In this case, the voice sales data is a sales conversation generated between the sales call agent and the customer through the sales call. The input unit 10 may acquire voice sales data by communicating with at least one of a counselor terminal and a customer terminal.

The control unit 12 converts the input voice sales data into text sales data (hereinafter referred to as 'text sales data') using voice recognition technology, and uses artificial intelligence technology for the converted text sales data. and analysis, extracts the key information required for sales call agent evaluation, calculates the call grade using the extracted core information, and recommends the calculated call grade. A detailed configuration of the control unit 12 will be described with reference to FIG. 2 .

The output unit 14 displays a screen provided according to the operation of the data analysis device 1 . In this case, the analysis result may be displayed through the control unit 12 .

The storage unit 16 stores information required for analysis by the control unit 12 and information generated according to the analysis result. For example, voice sales data, text sales data, a dictionary for information extraction, core information, call grade calculation results, and the like may be stored.

2 is a diagram illustrating a detailed configuration of the control unit of FIG. 1 according to an embodiment of the present invention.

1 and 2 , the control unit 12 includes an STT processing unit 121 , a dictionary construction unit 122 , an information extraction unit 123 , a call grade calculation unit 124 , and a call grade recommendation unit 125 . include

The STT processing unit 121 converts voice sales data generated through an inside sales call of an agent into text sales data.

The dictionary construction unit 122 builds a dictionary for information extraction from the text sales data converted through the STT processing unit 121 . The dictionary building unit 122 builds a dictionary of entity names corresponding to words belonging to a predetermined set among the core vocabulary of the customer, and constructs a semantic dictionary corresponding to common nouns for grasping the meaning of sentences other than entity names, A dialogue act dictionary corresponding to a sentence may be constructed to determine whether the customer is uttering with one of a question, an answer, a general, and a solicitation intention. In addition, a pattern dictionary corresponding to the result of synthesizing the entity name dictionary, the semantic dictionary, and the dialogue act dictionary can be constructed. In this case, the dictionary building unit 122 assigns a representative keyword to the result of synthesizing the entity name dictionary, the semantic dictionary, and the dialogue act dictionary, and the representative keyword may be composed of keywords representing BANT information. An example of the dictionary construction of the dictionary construction unit 122 will be described later with reference to FIG. 5 .

The information extraction unit 123 extracts core information for call rating calculation from the text sales data using the dictionary built through the dictionary construction unit 122 . For example, the information extraction unit 123 refers to dictionaries for information extraction built through the dictionary construction unit 122 and provides core information according to an index required for call rating calculation (hereinafter referred to as a 'call rating calculation index'). , and calculate the sound source playback time. The call rating calculation index may include at least one of a sales conversation time, a question score compared to a sales conversation time, a BANT weighted question score, a solicitation question score, and the number of keywords for each BANT. An example of extracting core information through the information extraction unit 123 will be described later with reference to FIG. 7 .

Artificial intelligence may be used to extract core information of the information extraction unit 123 . To this end, the control unit 12 builds learning data for extracting sales information by using the text sales data converted through the STT processing unit 121, and a learning unit (not shown) that generates a learning model for the constructed learning data. may further include. The training data may be deep learning training data, and the training model may be a deep learning training model. The information extraction unit 123 may extract core information from the text sales data by using a deep learning learning model generated through the learning unit.

The call grade calculation unit 124 calculates a grade for the inside sales call of the agent using the core information extracted by the information extraction unit 123 , and the call grade recommendation unit 125 recommends the calculated call grade. For example, the call grade calculation unit 124 calculates the final call grade value through arithmetic operation of the standard for each index, which is the standard for each project, and the weight for each index with respect to the core information extracted through the information extraction unit 123 . The call grade recommendation unit 125 recommends a call grade according to the call grade index with the final value calculated by the call grade calculation unit 124 .

The standard for each index is a differential classification of core information from each call grade calculation index, and the weight for each index indicates the importance of each call grade calculation index, and values can be distributed based on the sum of 1. The call grade classification index may be a call grade differentially assigned to a final value obtained by arithmetic operation of a criterion for each index and a weight for each index.

An example of the call rating calculation by the call rating calculator 124 and the call recommendation by the call rating recommender 125 will be described later with reference to FIG. 8 .

3 is a diagram illustrating a flow of a method for evaluating a call agent according to an embodiment of the present invention.

1 to 3 , the data analysis apparatus 1 converts voice sales data generated through an inside sales call of an agent into text sales data ( 310 ). Step 310 may be performed through the STT processing unit 121 of FIG.

Next, the data analysis device 1 builds a dictionary for information extraction from the converted text sales data ( 320 ). Dictionaries for information extraction include entity name dictionary, semantic dictionary, dialogue act dictionary, and pattern dictionary. Step 320 may be performed through the dictionary construction unit 122 of FIG. 2 .

Next, the data analysis device 1 extracts core information required for call rating calculation from the text sales data using the built-up dictionary ( 330 ). Key information is indicators used for inside sales call agent evaluation, and includes sales conversation time, question score compared to sales conversation time, BANT weighted question score, solicitation question score, and number of keywords for each BANT. BANT information is representative sales information that can be extracted from sales conversations with customers. It was first proposed by IBM as a criterion for identifying sales opportunities and has become the standard of sales. BANT information is used for text information by dividing it into information such as budget (Budget: B), authority (Authority: A), needs (Needs: N), and time (Time frame: T) required for sales. Step 330 may be performed through the information extraction unit 123 .

Next, the data analysis apparatus 1 calculates a grade for the inside sales call of the agent by using the extracted core information ( 340 ). This step may be performed through the call rating calculator 124 .

Next, the data analysis device 1 recommends the calculated call grade ( 350 ). This step may be performed through the call rating recommendation unit 125 .

4 is a diagram illustrating an STT recognition process according to an embodiment of the present invention.

2 and 4, the STT processing unit 121 receives the customer's speech (Speech) sales data, converts it into text (TXT) sales data using STT (Speech-To-Text) technology, and then outputs it .

In order to maximize the recognition rate of the STT processing unit 121, it is essential to learn the domain to be applied. For this, you need a customer's voice sales data file and a transcription file for it. The customer's voice sales data file is a file in which the conversation with the customer is recorded. The transcription file is the content of the corresponding recorded file, and is a file in which the audio file is converted into text as it is. When the domain learning of the STT processing unit 121 is performed using the customer's speech sales data file and the transcription file, an acoustic model and a language model specialized for the domain are generated. . The STT processing unit 121 converts the customer's voice sales data into text sales information using an acoustic model and a language model specialized for the domain used.

5 is a diagram illustrating an example of a dictionary for extracting information according to an embodiment of the present invention, and FIG. 6 is a diagram illustrating an example of BANT information and representative keywords according to an embodiment of the present invention.

2, 5 and 6 , the dictionary building unit 122 builds a dictionary for information extraction by using the converted text sales information. There are 1) Entity Name Dictionary, 2) Semantic Dictionary, 3) Dialog Act Dictionary, and 4) Pattern Dictionary to be built for information extraction.

1) The entity name dictionary is a word belonging to a specific set among the customer's core vocabulary. Add '@' tag to the object name. For example, it can be used to analyze a job title, company name, product name, etc. as an entity name.

2) The semantic dictionary is a common noun that plays an important role in understanding the meaning of sentences other than the name of the entity. Add '$' tag to common nouns. As an example, introduction, purchase, and schedule are handled as a semantic dictionary.

3) A dialogue act dictionary is a sentence that can identify what the customer intends to utter, and is used to analyze dialogue acts. A dialogue act consists of one of the intentions of question, answer, general, and solicitation. When a question is asked to the other party, an answer is an answer to a question, and a general is a declarative text, the solicitation is made by a salesperson (agent) to the customer. It represents the form of making a request such as “Would you like to” or “I would like to ask you”. To classify dialogue acts, '#' tags are assigned to distinguish intentions.

4) The pattern dictionary is the result of synthesizing the entity name, semantic, and dialogue act dictionary, and it is separated through '/', and a keyword (representative keyword) that can represent it is given. In the example of FIG. 5 , “Would you like to deliver the data requested by the team leader” can be expressed as “@POSITION/$delivery/#question=personal interest”, and the representative keyword here is 'personal interest'.

The representative keyword is composed of keywords representing BANT information, and the keywords for this are the same as cost, person in charge, personal interest, interest, etc. The role of the pattern dictionary is used to extract a representative keyword representing BANT information from a sentence that matches the pattern with the text information converted by the STT processing unit 121 . The extracted representative keyword is referred to in order to derive core information from the information extraction unit 123 .

7 is a diagram illustrating an example of core information required for call class calculation according to an embodiment of the present invention.

Referring to FIGS. 2 and 7 , the information extraction unit 123 extracts 'core information' according to the call rating calculation index with reference to the built-up information extraction dictionaries, and calculates the sound source reproduction time. The 'call rating calculation index' includes 1) sales conversation time, 2) question score compared to sales conversation time, 3) BANT weighted question score, 4) Kwon type question score, and 5) number of keywords for each BANT. 1) The sales conversation time is the reproduction time of the audio sales information data file. For example, the playback time is 463 seconds, and '463 seconds' is key information.

2) The question score compared to the sales conversation time is the result of dividing the total number of questions by the sales conversation time. In the case of the number of questions, it is the number of tag information matched to '#question' in the pattern dictionary of the dictionary building unit. For example, if the playback time is 463 and the total number of questions is 35, the question score compared to the sales conversation time is 35 / 463 = 7.5, and '7.5' is the key information.

3) The BANT weighted question score is the sum of the product of the weight assigned to each BANT and the number of questions divided by the total number of questions. For example, the weights for each BANT are B (10%), A (30%), N (50%), T (10%), and the number of questions per BANT is B (0 times), A (3 times), If N(16), T(1), the BANT weighted question score is (0*0.1) +(3*0.3) + (16*0.5) +(1*0.1) / 35 = 0.26, and '0.26 'This is key information.

4) The recommendation type question score is the number of tag information matched with '#recommendation' in the pattern dictionary of the dictionary building unit 122 . For example, the number of recommended questions: 4, and '4' is the key information.

5) The number of BANT keywords is the number of representative keywords returned when text information separated by BANT and the pattern dictionary are matched. For example, the number of representative keywords: 35, and '35' becomes the core information.

8 is a diagram illustrating an example of a call grade recommendation result according to a call grade calculation according to an embodiment of the present invention.

2 and 8 , the call grade calculation unit 124 applies an arithmetic operation of the standard for each index, which is the standard for each project, and the weight for each index, to the core information extracted by the information extraction unit 123 to obtain the final call grade value. to derive The call grade recommendation unit 125 uses the final value calculated by the call grade calculation unit 124 to recommend a call grade according to a project-specific standard.

Standards for each project include 1) standards for each indicator, 2) weights for each indicator, and 3) call grade classification indicators, and expert heuristic rules can be used. For each domain (project), 1) criteria for each indicator, 2) weight for each indicator, and 3) call grade classification indicators may have different values.

1) The criteria for each indicator differentially classifies the core information from each call grade calculation indicator and facilitates the calculation for call grade recommendation. 2) The weight for each indicator indicates the importance of each indicator, and the value is distributed based on the sum of 1. 3) The call grade classification index gives different call grades to the final value obtained by arithmetic operation of the standard for each indicator and the weight for each indicator. For example, the final value is (4.0*0.1) + (4.0*0.1) + (1.0*0.3) + (4.0*0.5) = 3.1, and the recommended call grade value is 'A'.

So far, the present invention has been focused on the embodiments thereof. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention can be implemented in modified forms without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims (8)

STT conversion unit for converting voice sales data generated through the agent's inside sales call to text sales data;
a dictionary building unit for constructing a dictionary for extracting information from the converted text sales data;
an information extraction unit for extracting core information for call rating calculation from text sales data using the built-in dictionary;
a call rating calculator for calculating a rating for an inside sales call of an agent by using the extracted core information; and
a call class recommendation unit that recommends the calculated call class;
Data analysis device comprising a. The method of claim 1, wherein the dictionary construction unit
constructing a dictionary of entity names corresponding to words belonging to a predetermined set among the customer's core vocabulary;
A semantic dictionary corresponding to common nouns is built to understand the meaning of sentences other than entity names,
Construct a dialogue act dictionary corresponding to a sentence to understand whether the customer is uttering with any intention among questions, answers, general, and solicitation;
A data analysis device characterized by constructing a pattern dictionary corresponding to the result of synthesizing the entity name dictionary, the semantic dictionary, and the dialogue act dictionary. The method of claim 2, wherein the dictionary construction unit
A representative keyword is given to the result of the synthesis of the entity name dictionary, semantic dictionary, and dialogue act dictionary.
The representative keyword is a data analysis device, characterized in that it is composed of keywords representing the BANT information. The method of claim 1, wherein the information extraction unit
A data analysis device characterized by extracting core information according to an index required for call rating calculation (hereinafter referred to as a 'call rating calculation index') with reference to the built-up dictionaries for information extraction, and calculating the sound source playback time. 5. The method of claim 4,
The call rating calculation index is a data analysis device, characterized in that it includes at least one of a sales conversation time, a question score compared to a sales conversation time, a BANT weight application question score, a solicitation question score, and the number of keywords for each BANT. The method of claim 1, wherein the call grade calculation unit
For the extracted core information, the final call grade value is calculated through the arithmetic operation of the standard for each index, which is the standard for each project, and the weight for each index.
The call grade recommendation unit has a final value calculated through the call grade calculation unit and recommends a call grade according to the call grade index. 7. The method of claim 6,
The standard for each indicator is the differential classification of core information from each call rating calculation indicator,
The weight for each indicator indicates the importance of each call rating calculation indicator, and the value is distributed based on the sum of 1.
The call grade classification index is a data analysis device, characterized in that the call grade is differentially assigned to the final value obtained by arithmetic operation of the standard for each index and the weight for each index. In the inside sales call agent evaluation method using the data analysis device, the data analysis device
converting voice sales data generated through an agent's inside sales call into text sales data;
building a dictionary for extracting information from the converted text sales data;
extracting core information for call rating calculation from text sales data using the built-in dictionary;
calculating a rating for an inside sales call of an agent by using the extracted core information; and
recommending the calculated call grade;
Inside sales call agent evaluation method comprising a.

Images