KR20230115723A - Real time consulting evaluation system - Google Patents

Abstract

A real-time counseling evaluation system according to the present invention includes an input interface and a processor. The processor performs analysis for each evaluation item, including the syllable agreement rate between the spoken sentence according to the real-time conversation and the evaluation item, and the speech time compliance rate based on the registered counseling quality evaluation target information, and when the real-time conversation progresses, the counseling application Information on evaluation items to be observed by the counselor may be exposed on one side of the counseling screen, and compliance or non-compliance with the evaluation items according to the real-time conversation may be displayed.

Description

Real time consulting evaluation system {Real time consulting evaluation system}

The present invention relates to a counseling system for providing a counseling service by a counselor. More specifically, the invention relates to a real-time counseling evaluation system. The present invention is a research conducted with the support of the Information and Communication Planning and Evaluation Institute with financial resources from the government (Ministry of Science and ICT) in 2021 (No.2021-0-00052, development and commercialization of an intelligent platform for contact center counseling assistants)

Recently, services based on advanced technologies such as artificial intelligence and robots are emerging like a flood due to the 4th industrial revolution. In fact, many people talk about what artificial intelligence can do, and even replacing the work of call center counselors is being discussed.

In addition, as various counseling channels have emerged in addition to telephone counseling, the term contact center has become more natural than the term call center.

On the other hand, in the past, the counseling manager manually evaluated the quality of counseling while listening to the recorded files at the end of each month. However, this has a disadvantage in that it takes a long time to listen to the recorded file, so that only some random call evaluations are possible, not all evaluations.

In particular, when new counselors with low understanding of work omit essential counseling items and the quality of counseling deteriorates, the problem arises of being subject to complaints from customers.

Therefore, the present invention intends to provide a real-time counseling evaluation system capable of evaluating all calls under the same criteria.

More specifically, it has the advantages of reducing analysis management time and human input cost, enabling call quality control over the entire call center, and enabling self-coaching for quality improvement as counselors can immediately see the results of their own calls.

Accordingly, an object of the present invention is to provide a real-time counseling method and a counseling system.

In addition, an object of the present invention is to provide a real-time counseling evaluation system capable of evaluating all calls under the same criteria.

In addition, an object of the present invention is to provide quality improvement self-coaching by reducing analysis management time and human input costs, enabling call center overall call quality management, and allowing counselors to immediately see the results of their own calls.

A real-time counseling evaluation system according to the present invention includes an input interface configured to acquire a real-time conversation through a microphone and a processor operatively coupled to the input interface and controlling a counseling application program to be executed and displayed on a display. . The processor performs analysis for each evaluation item, including a syllable agreement rate between a spoken sentence according to the real-time conversation and an evaluation item, and a speech time compliance rate based on registered counseling quality evaluation target information, and when the real-time conversation progresses, the processor Information on evaluation items to be observed by the counselor may be exposed on one side of the consultation screen of the consultation application, and compliance or non-compliance with the evaluation items according to the real-time conversation may be controlled to be displayed.

According to an embodiment, the processor drives an STT engine to analyze packets of the real-time conversation, match the packets and standard scripts in units of syllables to compare accuracy, and rule-set associated with the evaluation items. ), it is possible to determine compliance with the evaluation items based on.

According to an embodiment, the processor compares accuracy by matching the original text string associated with the standard script and the pattern string associated with the real-time conversation in units of syllables by varying matching intervals for each stage, and the standard script is It is characterized in that it is a predefined counseling script.

According to an embodiment, the rule set associated with the evaluation item may include a first rule set used to analyze whether a feature morpheme is included in the real-time conversation; and a second set of rules used for analysis of conversation interruption, silence detection, whether a response language is used, and whether a nonverbal sentence is used. The processor compares the accuracy of the pattern string associated with the real-time conversation with the original text string based on the first rule set, and based on the first rule set and the second rule set, N evaluation items compliance can be assessed.

According to an embodiment, if the counseling evaluation score of the counselor is equal to or less than the threshold value, the processor receives updated first rule sets and second rule sets from the server, interworks with the server, and based on the first rule set, Comparing the accuracy of the pattern string associated with the real-time conversation with the original text string, interworking with the server and determining compliance with N evaluation items based on the first rule set and the second rule set; When essential vocabulary for counseling is not used among the N evaluation items, a standard script including the essential vocabulary may be displayed on the display in conjunction with the server.

According to an embodiment, the processor provides real-time evaluation results of K evaluation items having high importance among the N evaluation items according to whether the characteristic morpheme is included in each sentence unit based on the first rule set, and the consultation It is provided through the API of an application program, and when the consultation according to the real-time conversation is terminated, based on the first rule set and the second rule set, compliance with N evaluation items is determined, and the N evaluation items are determined. Evaluation results for each item and consultation quality evaluation result may be displayed on the display.

According to an embodiment, the KMP (Knowledge Management Platform) associated with the real-time counseling evaluation system includes a knowledge data learning model for learning knowledge data, a classification data learning model for learning by classifying counseling texts, and a script and evaluation criteria model. It may be configured to include a consultation evaluation model. The processor performs keyword pattern-based analysis for performing morpheme analysis, keyword analysis, natural language processing, and attribute classification of the real-time conversation using the consultation evaluation model, and uses the consultation evaluation model and matching rate analysis model to perform the real-time conversation analysis. Whether to use first and last greetings in conversational sentences, whether to use forbidden words, checking overlapping sections between customers and counselors, checking whether there are silent sections that the counselor does not utter, whether to use responsive words, whether to use spoken words when waiting for customers, customer It is possible to determine whether or not to use the essential information to be notified, and to analyze the matching rate according to the keyword pattern associated with the essential counseling item defined in the KMP using the matching rate analysis model.

According to an embodiment, when the counseling application program is executed and real-time counseling evaluation is performed, a keyword pattern-based analysis module and a KMP matching rate analysis module may be executed. The processor performs consultation evaluation based on the first information and the second information output from the keyword pattern-based analysis module and the KMP matching rate analysis module, and interworks with the AI-based analysis module and the deep learning module during the real-time consultation evaluation. Instead, the AI-based analysis module and the deep learning module can be interlocked when recommending real-time knowledge contents or classifying consultation types.

According to an embodiment of the present invention, a real-time counseling evaluation method and a counseling evaluation system may be implemented.

In addition, according to an embodiment of the present invention, it is possible to implement a real-time counseling evaluation system capable of evaluating all calls under the same criteria.

In addition, according to an embodiment of the present invention, it is possible to reduce analysis management time and human input cost, to manage overall call quality of a call center, and to immediately see the result of a call by the counselor himself, so that quality improvement self-coaching is possible.

1 is a configuration diagram of a real-time counseling evaluation system according to an embodiment of the present invention.
2 is a detailed configuration diagram of a real-time counseling evaluation system according to an embodiment of the present invention.
Figure 3 shows a flow chart of a real-time KMS recommendation method performed in the real-time counseling evaluation system according to the present invention.
4 shows a detailed model and module configuration for counseling-related real-time processing in the real-time counseling evaluation system according to the present invention.
5a to 5c show configurations related to differentiated technical features of the real-time counseling evaluation system according to the present invention.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are only illustrated for the purpose of explaining the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention It can be embodied in various forms and is not limited to the embodiments described herein.

Embodiments according to the concept of the present invention can apply various changes and have various forms, so the embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosure forms, and includes all changes, equivalents, or substitutes included in the spirit and technical scope of the present invention.

Terms such as first or second may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another, e.g. without departing from the scope of rights according to the concept of the present invention, a first component may be termed a second component and similarly a second component may be termed a second component. A component may also be referred to as a first component.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when a component is referred to as “directly connected” or “directly connected” to another component, it should be understood that no other component exists in the middle. Other expressions describing the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", etc., should be interpreted similarly.

Terms used in this specification are used only to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in this specification, but one or more other features It should be understood that it does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these examples. Like reference numerals in each figure indicate like elements.

1 is a configuration diagram of a real-time counseling evaluation system according to an embodiment of the present invention. As shown in FIG. 1 , the real-time counseling evaluation system according to an embodiment of the present invention is a system composed of a customer terminal 100 , a server 200 , a counselor computer 300 and a communication network 400 . The communication network 400 may be a wired and/or wireless communication network that communicatively connects the customer terminal 100 and the counselor computer 300 . In addition, the communication network 400 may be a wired and/or wireless communication network that communicatively connects the customer terminal 100 and the server 200 or the server 200 and the counselor computer 300 .

Meanwhile, FIG. 2 is a detailed configuration diagram of a real-time counseling evaluation system according to an embodiment of the present invention.

1 and 2, the counselor computer 300 may be configured to include a communication unit 310, an input interface 320, a processor 330, a display 340 and a memory 350. The server 200 is operably coupled with a plurality of customer terminals 100 and counselor computers 300 and may be composed of a plurality of servers.

The communication unit 310 may be configured to be communicatively coupled with a plurality of customer terminals 100 and the server 200, and may be implemented as a wired and/or wireless communication network. The input interface 320 may be configured to acquire real-time conversations of the counselor and/or the customer through the microphone 321 . The input interface 320 is not limited thereto and may further include other input means capable of controlling the counselor computer 300, such as a keyboard and a mouse.

Processor 330 may be operably coupled with input interface 32 . The processor 330 may control the counseling application program 600 to be executed so that the counseling screen is displayed on the display 340 . The memory 350 may be configured to store counseling-related information and to access the information when a counseling application program is executed.

The real-time counseling evaluation system according to the present invention may provide compliance by evaluating in real time the counselor's essential utterance sentences, usage of prohibited words, response delay time to customer inquiries, and the like during counseling. Previously, at the end of each month, the counseling manager listened to the recorded files and evaluated the quality of counseling manually. However, this has a disadvantage in that it takes a long time to listen to the recorded file, so that only some random calls can be evaluated rather than all evaluations.

Therefore, the present invention can evaluate all calls under the same criteria. Specifically, it has the advantages of reducing analysis management time and human input costs, enabling call quality control over the entire call center, and enabling self-coaching for quality improvement as counselors can immediately see the results of their own calls.

Implementation of the real-time consultation evaluation system according to the present invention is performed by registering the consultation quality evaluation target information in the system. For example, whether a first greeting or a final greeting is used, whether prohibited words are used, word overlap between the customer and the counselor, check whether the counselor does not utter a silent section, whether a response is used, whether a spoken word is used when waiting for a customer, and required for the customer Information to be notified may be subject information for counseling quality evaluation, but is not limited thereto.

In addition, in the implementation of the real-time counseling evaluation system according to the present invention, when a real-time conversation is in progress, information on evaluation items to be followed by the counselor is exposed on the right side of the call application screen, and compliance/non-compliance may be displayed while the counselor is conducting a conversation. there is.

In relation to the concept and implementation method of the present invention described above, the real-time counseling method of the present invention may be configured as shown in FIG. 3 . In this regard, FIG. 3 shows a flowchart of a real-time counseling method performed in the real-time counseling evaluation system according to the present invention.

Referring to FIG. 3 , the real-time counseling evaluation system may include an STT engine 540 and a counseling application program 600 .

The STT engine 540 may perform a matching rate and rule-set complex analysis by analyzing real-time packets according to real-time conversations. Regarding the matching rate and rule-set composite analysis, the dialog-standard script can be compared in accuracy based on syllable unit manager-defined matching. In addition, evaluation of administrator-defined items in units of evaluation rule sets may be performed.

The syllable unit manager-defined matching-based accuracy comparison can be performed through an analysis algorithm through KMP string matching and a predefined counseling script for each evaluation item. Evaluation of administrator-defined items in units of evaluation rule sets may be performed in consideration of whether feature morphemes are included and whether conversation interruption/silence detection/response words and avoidance statements are used.

Through such a matching rate and rule set composite analysis, an analysis result for N evaluation items, for example, compliance evaluation for each evaluation item may be performed.

Real-time results for each sentence-by-sentence evaluation item can be delivered to the counseling application program 600 in real time using an API. On the other hand, the final evaluation result in a call unit is made immediately after the end of counseling through the real-time results of each sentence unit evaluation item. In addition, the final evaluation result for each call can be delivered to the counseling application program 600 using an API.

In relation to this implementation, the real-time counseling evaluation system may be implemented through the counselor computer 300 of FIG. 2 or by the server 200 interworking with the counselor computer 300 .

Referring to FIGS. 2 and 3 , the processor 330 may be operably coupled with the input interface 320 . The processor 330 may control the counseling application program 600 to be executed so that the counseling screen is displayed on the display 340 .

The processor 330 may perform analysis for each evaluation item, including a syllable agreement rate between a spoken sentence according to real-time conversation and an evaluation item, and a speech time compliance rate, based on registered counseling quality evaluation target information. When a real-time conversation progresses, the processor 330 may control information on evaluation items to be observed by the counselor to be displayed on one side of the counseling screen of the counseling application 600 . The processor 330 may control compliance or non-compliance with evaluation items according to real-time conversation to be displayed.

The processor 330 may drive the STT engine 540 to analyze packets of real-time conversation, match the packets and standard scripts in units of syllables, and compare accuracy. The processor 330 may determine compliance with the evaluation item based on a rule-set associated with the evaluation item.

The processor 330 may compare the accuracy by matching the original text string related to the standard script and the pattern string related to the real-time conversation in units of syllables by changing matching intervals for each step. In this regard, the standard script may be configured as a predefined counseling script for each evaluation item.

A rule set associated with an evaluation item may be configured to include a first rule set and a second rule set. The first rule set may include rules used to analyze whether a feature morpheme is included in a real-time conversation. The second rule set may be used for barring in, silence detection, whether to use a response word, and whether to use a suppression sentence.

The processor 330 may compare accuracy of a pattern string associated with a real-time conversation with the original text string based on the first rule set. The processor 330 may determine compliance with N evaluation items based on the first rule set and the second rule set.

The processor 330 may compare accuracy of the pattern string associated with the real-time conversation with the original text string based on a first rule set. The processor 330 may determine compliance with N evaluation items based on the first rule set and the second rule set.

When the counseling evaluation score of the counselor is equal to or less than the threshold value, the processor 330 may receive updated first rule sets and second rule sets from the server 200 . Accordingly, it is possible to specially manage a counselor with a low consultation evaluation score, and it is possible to manage counseling on non-consultation items through counseling evaluation during real-time counseling. In addition, during real-time counseling, it is possible to manage to improve the quality of real-time counseling by determining whether to interrupt a conversation, detect silence, use an answering word, or not use a negative comment. To manage consultation items in real time and improve consultation quality, the processor 330 may receive, from the server 200, first and second rule sets that are updated in consideration of individual characteristics of each consultation.

Accordingly, the processor 330 may compare the accuracy of the pattern string associated with the real-time conversation with the original text string based on the first rule set in association with the server 200 . The processor 330 may interwork with the server 200 to determine compliance with N evaluation items based on the first rule set and the second rule set. If essential vocabulary for counseling is not used among the N evaluation items, the processor 330 may display a standard script including essential vocabulary on the display 340 in conjunction with the server 200 .

The processor 330 transfers real-time evaluation results of K evaluation items with high importance among the N evaluation items according to whether the feature morpheme is included in each sentence based on a first rule set to the counseling application program 600. It can be provided through the API of When counseling based on real-time conversation ends, the processor 330 may determine compliance with N evaluation items based on the first rule set and the second rule set. Accordingly, the processor 330 may display the evaluation results for each N number of evaluation items and the consultation quality evaluation result on the display 340 .

Meanwhile, various learning models and evaluation criterion models, including the counseling evaluation model according to the present invention, may be implemented for real-time processing related to counseling. In this regard, FIG. 4 shows a detailed model and module configuration for counseling-related real-time processing in the real-time counseling evaluation system according to the present invention.

2 to 4 , when the counseling application program 600 is driven by the processor 330, the counseling application program 600 may be layered as follows. As another example, the counseling application program 600 may be implemented in the server 200 .

The counseling application program 600 is configured to include an input interface 610, a learning/evaluation model 620, a keyword pattern-based analysis module 630, an AI-based analysis module 640, and a real-time processing/interworking module 650. It can be. The keyword pattern-based analysis module 630 may include the KMP matching rate analysis module 631 or the matching rate analysis module 631 may be configured separately from the keyword pattern-based analysis module 630 . The AI-based analysis module 640 may include the deep learning module 641 or the deep learning module 641 may be configured separately from the AI-based analysis module 640 .

The input interface 610 may be configured to obtain consultation text, application metadata, KMS-related data, and administrator-defined data. The learning/evaluation model 620 may be referred to as a KMS learning model. The learning/evaluation model 620 may include a knowledge data learning model 621 that learns knowledge data. The learning/evaluation model 620 may be configured to include a classification data learning model 622 that classifies and learns original counseling texts and a counseling evaluation model 623 including a script and an evaluation criterion model.

KMP (Knowledge Management Platform) associated with the real-time counseling evaluation system, as described above, includes a knowledge data learning model 621 that learns knowledge data, a classification data learning model 622 that learns by classifying original counseling texts, and a script and evaluation criterion model It may be configured to include a counseling evaluation model 623 including a.

The processor 330 may perform keyword pattern-based analysis to perform morphological analysis, keyword analysis, natural language processing, and attribute classification of the real-time conversation using the counseling evaluation model 623 . The processor 330 may determine compliance with evaluation items by using the counseling evaluation model 623 and the matching rate analysis model. The processor 330 checks whether the first greeting and the last greeting are used in a sentence according to real-time conversation, whether prohibited words are used, checks overlapping sections between the customer and the counselor, checks whether there is a silent section that the counselor does not utter, whether a response is spoken, and when a customer requests to wait It is possible to determine whether or not to use the spoken word and whether to use the necessary notice to the customer. The processor 330 may analyze a matching rate according to a keyword pattern associated with essential counseling items defined in the KMP using a matching rate analysis model.

Meanwhile, when the counseling application program 600 is executed and real-time counseling evaluation is performed, the keyword pattern-based analysis module 630 and the KMP matching rate analysis module 631 may be executed. The processor 330 may perform counseling evaluation based on the first information and the second information output from the keyword pattern-based analysis module 630 and the KMP matching rate analysis module 631 . The processor 330 does not interwork with the AI-based analysis module 640 and the deep learning module 641 during real-time counseling evaluation, and the AI-based analysis module 640 and the deep learning module 641 recommend real-time knowledge content or consult types. It can be controlled to interlock during classification.

Therefore, the real-time counseling evaluation method by the real-time counseling evaluation system according to the present invention may be performed by a combination of 1) + 4) + 5) + 6) + 8), but is not limited thereto.

On the other hand, the processor 330 can guarantee the accuracy of real-time counseling evaluation through the application of the AI analysis engine's differentiation technology. In this regard, FIGS. 5A to 5C show configurations related to differentiated technical features of the real-time counseling evaluation system according to the present invention. 5a shows a configuration for applying a BERT + GPT mixed model. 5B shows a configuration for combining byte pair encoding and morpheme analysis during conversion and tokenization of written language data. 5C shows a configuration to which a string matching improvement algorithm considering free speech characteristics is applied. To this end, a sentence search screen area within the KMP algorithm is defined, and accordingly, a string matching improvement algorithm considering free speech characteristics may be implemented.

Referring to FIGS. 2 to 5A , the processor 330 may apply a weight obtained through a Bidirectional Encoder Representations from Transformers (BERT) method to a Generative Pre-trained Transformer (GPT) method.

The processor 330 may perform decoding on each sentence associated with the counseling contents, morpheme analysis, keyword analysis, natural language processing, and attribute classification. By performing decoding on each sentence related to the counseling content through a configuration that applies the BERT + GPT mixed model, morphological analysis, keyword analysis, natural language processing and attribute classification, improvement of knowledge content recommendation and / or accuracy of real-time counseling evaluation and Speed improvement is possible.

In the above, the real-time counseling evaluation system according to the present invention has been examined. The technical effects of the real-time counseling evaluation system according to the present invention are as follows.

According to an embodiment of the present invention, a real-time counseling method and counseling system can be implemented.

In addition, according to an embodiment of the present invention, it is possible to implement a real-time counseling evaluation system capable of evaluating all calls under the same criteria.

In addition, according to an embodiment of the present invention, it is possible to reduce analysis management time and human input cost, to manage overall call quality of a call center, and to immediately see the result of a call by the counselor himself, so that quality improvement self-coaching is possible.

The features and effects of the present invention described above will become more apparent through the following detailed description in conjunction with the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs can easily practice the technical idea of the present invention. You will be able to.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

According to the software implementation, not only the procedures and functions described in this specification, but also the design and parameter optimization of each component may be implemented as a separate software module. The software code may be implemented as a software application written in any suitable programming language. The software code may be stored in a memory and executed by a controller or processor.

100: customer terminal
200: server
300: counselor computer
310: communication department
320: input interface
330: processor
340: display
350: memory
400: communication network
540: STT engine
600: counseling application program
610: input interface
620: learning/evaluation model
621: knowledge data learning model
622: classification data learning model
623: consultation evaluation model
630: keyword pattern based analysis module
640: AI-based analysis module
650: real-time processing / interlocking module

Claims (8)

In the real-time consultation evaluation system,
an input interface configured to acquire a real-time conversation through a microphone; and
a processor operably coupled to the input interface and controlling a counseling application program to be executed and displayed on a display;
the processor,
Based on the registered counseling quality evaluation target information, analysis is performed for each evaluation item, including the syllable agreement rate between the spoken sentence according to the real-time conversation and the evaluation item, and the speech time compliance rate,
When the real-time conversation proceeds, information on evaluation items to be observed by the counselor is exposed on one side of the consultation screen of the consultation application, and compliance or non-compliance with the evaluation items according to the real-time conversation is displayed. rating system. According to claim 1,
the processor,
The STT engine is driven to analyze packets of the real-time conversation, match the packets and standard scripts in units of syllables to compare accuracy, and comply with the evaluation items based on rule-sets associated with the evaluation items. A real-time consultation evaluation system that determines whether or not According to claim 2,
the processor,
Compare accuracy by matching the original text string associated with the standard script and the pattern string associated with the real-time conversation in units of syllables by varying the matching section for each step,
The standard script is a counseling script predefined for each evaluation item, characterized in that, real-time counseling evaluation system. According to claim 3,
The rule set associated with the evaluation item,
a first set of rules used to analyze whether a feature morpheme is included in the real-time conversation; and
A second set of rules used to analyze conversation interruption, silence detection, response language use and suppression use,
the processor,
Based on the first set of rules, compare accuracy of a pattern string associated with the real-time conversation with the textual string;
Based on the first rule set and the second rule set, determining compliance with N evaluation items, real-time counseling evaluation system. According to claim 4,
the processor,
If the counseling evaluation score of the counselor is less than or equal to the threshold value, receiving an updated first rule set and a second rule set from the server;
Comparing accuracy of a pattern string associated with the real-time conversation with the original text string based on the first rule set in conjunction with the server;
In conjunction with the server, based on the first rule set and the second rule set, determining compliance with N evaluation items;
The real-time counseling evaluation system, which displays a standard script including the essential vocabulary on the display in conjunction with the server when essential vocabulary for counseling is not used among the N evaluation items. According to claim 4,
the processor,
Based on the first rule set, real-time evaluation results of K evaluation items with high importance among the N evaluation items according to whether the characteristic morpheme is included in each sentence unit are provided through the API of the counseling application program,
When the counseling according to the real-time conversation is terminated, based on the first rule set and the second rule set, compliance with N evaluation items is determined, and the evaluation result for each of the N evaluation items and the consultation quality evaluation result A real-time consultation evaluation system displayed on the display. According to claim 5,
The KMP (Knowledge Management Platform) associated with the real-time counseling evaluation system includes a knowledge data learning model for learning knowledge data, a classification data learning model for learning by classifying original counseling texts, and a counseling evaluation model including scripts and evaluation criteria models. constituted,
the processor,
Perform keyword pattern-based analysis to perform morpheme analysis, keyword analysis, natural language processing, and attribute classification of the real-time conversation using the counseling evaluation model;
Using the counseling evaluation model and the matching rate analysis model, in the sentences according to the real-time conversation, whether to use first and last words, whether to use prohibited words, check overlapping sections between the customer and the counselor, check whether the counselor does not utter a silent section, and respond. Determine whether the language is spoken, whether the spoken word is used when a customer is waiting for a request, and whether the required notice is used to the customer
A real-time counseling evaluation system that analyzes a matching rate according to a keyword pattern associated with an essential counseling item defined in the KMP using the matching rate analysis model. According to claim 1,
When the counseling application program is executed and real-time counseling evaluation is performed, a keyword pattern-based analysis module and a KMP matching rate analysis module are executed.
the processor,
Consulting evaluation is performed based on first information and second information output from the keyword pattern-based analysis module and the KMP matching rate analysis module;
Characterized in that the AI-based analysis module and the deep learning module do not interwork with the AI-based analysis module and the deep learning module during the real-time consultation evaluation, and the AI-based analysis module and the deep learning module are interlocked when recommending real-time knowledge contents or classifying consultation types. Real-time consultation evaluation system.

Images