JP2021012303A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

To provide an information processing device, an information processing method and an information processing program which have improved convenience.SOLUTION: In an information processing system, a CPU 500C of an information processing device 5 (server) comprises: a voice information acquisition unit which acquires voice information on a first speaker and a second speaker to have conversation with the first speaker; a feeling analysis unit which analyzes the feeling of the first speaker and the second speaker from the voice information acquired by the voice information acquisition unit; an evaluation unit which evaluates a response to the first speaker by the second speaker depending on the analysis result in the feeling analysis unit; and a presentation unit which presents the evaluation result in the evaluation unit.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Systems for supporting the operations of call centers have been conventionally proposed. For example, in Patent Document 1, in an inbound call center system such as a financial institution, by analyzing the conversation content between a customer and a virtual operator or an operator, it is detected that the customer is in a complaint state during a customer call and the situation worsens. A call center system with a preventive function has been proposed.

JP-A-2019-047170

However, in the conventional invention, there is room for improvement in convenience, such as being unable to prevent a customer from becoming a complaint.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an information processing device, an information processing method, and an information processing program having more convenience.

In order to solve the above problems, the information processing device of the present invention is acquired by a voice information acquisition unit that acquires voice information of a first speaker and a second speaker who talks with the first speaker, and a voice information acquisition unit. An evaluation that evaluates the response of the second speaker to the first speaker according to the analysis results of the emotion analysis unit that analyzes the emotions of the first speaker and the second speaker from the voice information that has been generated. It includes a unit and a presentation unit that presents the evaluation results of the evaluation unit.

According to the present invention, it is possible to provide an information processing device, an information processing method, and an information processing program that are more convenient.

It is a figure which shows an example of the schematic structure of the information processing system which concerns on embodiment. It is a figure which shows an example of the structure of the OP terminal which concerns on embodiment. It is a figure which shows an example of the structure of the SV terminal which concerns on embodiment. It is a figure which shows an example of the configuration of the server which concerns on embodiment. This is an example of a display screen displayed on the display device of the OP terminal according to the embodiment. It is a flowchart which shows an example of the main processing by the information processing system which concerns on embodiment. It is a flowchart which shows an example of the emotion analysis processing by the information processing system which concerns on embodiment. It is a flowchart which shows an example of the evaluation presentation processing by the information processing system which concerns on embodiment. It is a flowchart which shows an example of the talk script presentation processing by the information processing system which concerns on embodiment. It is a flowchart which shows an example of the advice presentation processing by the information processing system which concerns on embodiment. It is a flowchart which shows an example of SV presentation processing by the information processing system which concerns on embodiment. It is a figure which shows an example of the functional structure of the server which concerns on the modification 1 of embodiment. It is a flowchart which shows an example of correspondence information presentation processing by the information processing system which concerns on modification 1 of embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, an example in which the present invention is applied to a call center is described, but the present invention is not limited to the call center and can be applied.

[Embodiment]
(Configuration of information processing system 1)
As shown in FIG. 1, the information processing system 1 of the embodiment is an IP-PBX (Internet Protocol-Private Branch eXchange, IP) connected from a call terminal 2A of a customer CS (first speaker) via a public network 7. A line-compatible private branch exchange) device 6 (hereinafter, also referred to as a PBX device 6) has a configuration in which it connects to a call terminal 2B of an operator OP (second speaker) via a network 8 to make a call. Further, an operator terminal 3 (hereinafter, also referred to as OP terminal 3), a supervisor terminal 4 (hereinafter, also referred to as SV terminal 4), and a server 5 (information processing device) are connected to the network 8 so as to be able to communicate. There is. The number of the call terminals 2A and 2B, the OP terminal 3, the SV terminal 4, the server 5, and the PBX device 6 included in the information processing system 1 is arbitrary.

(Configuration of OP terminal 3)
The OP terminal 3 is a terminal used by the operator OP, and is, for example, a desktop PC or a notebook PC. As shown in FIG. 2A, the OP terminal 3 includes a communication IF 300A, a storage device 300B, an input device 300C, a display device 300D, and a CPU 300E.

The communication IF300A is an interface for communicating with other devices.

The storage device 300B is, for example, an HDD (Hard Disk Drive) or a semiconductor storage device (SSD (Solid State Drive)), and for example, an information processing program, a terminal ID, or the like is stored. The terminal ID is an identifier for identifying the OP terminal 3. By assigning a terminal ID to the information transmitted from the OP terminal 3, the server 5 can determine from which OP terminal 3 the received information is transmitted. The terminal ID may be an IP (Internet Protocol) address, a MAC (Media Access Control) address, or the like, or may be assigned by the server 5 to the OP terminal 3.

The input device 300C is, for example, an input device such as a keyboard or a mouse, but may be another device or device (for example, a touch panel) as long as input is possible.

The display device 300D is, for example, a liquid crystal display, a plasma display, an organic EL display, or the like, but may be another device or device (for example, CRT: Cathode Ray Tube) as long as it can display.

The CPU 300E controls the OP terminal 3. The CPU 300E includes a ROM (Read Only Memory) and a RAM (Random Access Memory) (not shown).

In the present embodiment, the OP terminal 3 is configured to include the storage device 300B, but another device (for example, a server) connected via the network 8 is configured to include all or a part of the storage device 300B. It may be. In this case, the OP terminal 3 refers to the storage device 300B included in the other device. Further, the information processing program may be configured to be downloadable from another device connected via the network 8.

(Functional configuration of OP terminal 3)
As shown in FIG. 2B, the OP terminal 3 has functions such as a receiving unit 301, a storage device control unit 302, an input receiving unit 303, a display device control unit 304, and a transmitting unit 305.

The receiving unit 301 receives the information transmitted from the server 5.

The storage device control unit 302 controls the storage device 300B. Specifically, the storage device control unit 302 controls the storage device 300B to write and read information.

The input receiving unit 303 receives an input operation from the input device 300C by the operator OP.

The display device control unit 304 controls the display device 300D.

The transmission unit 305 transmits the information of the input operation received by the input reception unit 303 to the server 5.

(Configuration of SV terminal 4)
The SV terminal 4 is a terminal used by the supervisor SV who is the administrator of the operator OP, and is preferably a portable tablet PC or the like. By making the SV terminal 4 a portable tablet PC, the supervisor SV can carry the SV terminal 4 around in the call center, which is excellent in convenience. The SV terminal 4 may be a device other than a tablet PC as long as it is portable.

As shown in FIG. 3A, the SV terminal 4 includes a communication IF 400A, a storage device 400B, an input device 400C, a display device 400D, and a CPU 400E.

The communication IF400A is an interface for communicating with other devices.

The storage device 400B is, for example, an HDD (Hard Disk Drive) or a semiconductor storage device (SSD (Solid State Drive)), and for example, an information processing program, a terminal ID, or the like is stored. The terminal ID is an identifier for identifying the SV terminal 4. By assigning a terminal ID to the information transmitted from the SV terminal 4, the server 5 can determine from which SV terminal 4 the received information is transmitted. As the terminal ID, an IP (Internet Protocol) address, a MAC (Media Access Control) address, or the like may be used, or the server 5 may assign the terminal ID to the SV terminal 4.

The input device 400C is, for example, an input device such as a touch panel, but may be another device or device (for example, a keyboard) as long as input is possible.

The display device 400D is, for example, a liquid crystal display, a plasma display, an organic EL display, or the like, but may be another device or device as long as it can display.

The CPU 400E controls the SV terminal 4. The CPU 400E includes a ROM (Read Only Memory) and a RAM (Random Access Memory) (not shown).

In the present embodiment, the SV terminal 4 is configured to include the storage device 400B, but another device (for example, a server) connected via the network 8 is configured to include all or a part of the storage device 400B. It may be. In this case, the SV terminal 4 refers to the storage device 400B provided in another server. Further, the information processing program may be configured to be downloadable from another server connected via the network 8.

(Functional configuration of SV terminal 4)
As shown in FIG. 3B, the SV terminal 4 has functions such as a receiving unit 401, a storage device control unit 402, an input receiving unit 403, a display device control unit 404, and a transmitting unit 405.

The receiving unit 401 receives the information transmitted from the server 5.

The storage device control unit 402 controls the storage device 400B. Specifically, the storage device control unit 302 controls the storage device 400B to write and read information.

The input receiving unit 403 receives an input operation by the input device 400C.

The display device control unit 404 controls the display device 400D.

The transmission unit 405 transmits the information of the input operation received by the input reception unit 403 to the server 5.

(Configuration of server 5)
As shown in FIG. 4A, the server 5 includes a communication IF 500A, a storage device 500B, and a CPU 500C.

The communication IF500A is an interface for communicating with other devices.

The storage device 500B is, for example, an HDD (Hard Disk Drive) or a semiconductor storage device (SSD (Solid State Drive)), and stores various databases (hereinafter, also referred to as DB), information processing programs, and the like. The storage device 500B may be another device or device as long as information (data) can be written and read. In the present embodiment, the server 5 includes the storage device 500B, but it may be stored in a computer-readable external storage medium (for example, a USB memory or an external HDD), or may be stored via the network 8. It may be configured to be readable and / or writable from another computer, server, or the like.

(Storage device)
The first DB to the sixth DB are stored in the storage device 500B. Hereinafter, the information stored in each DB will be described. In the present embodiment, the first DB to the sixth DB are stored in the storage device 500B, but the DB structure does not necessarily have to be stored in the storage device 500B.

(1st DB (1st storage unit))
In the first DB, the stage of responding to the customer CS by the operator OP (talk phase, for example, sales, opening, offer, rebattle, closing, etc.) and the talk script related to the stage are stored in association with each other. Here, the talk script is an explanatory wording about a product or a contract content in outbound sales or inbound sales at a call center, and the explanatory wording is associated and stored for each stage (talk face).

(2nd DB)
Advice to the operator OP is stored in the second DB (second storage unit). In the present embodiment, the advice is stored in the second DB as a casebook, and the method of dealing with the customer CS corresponds as the advice for each case (for example, the customer is angry, the complaint is spoken endlessly, etc.). Is stored in the second DB. The advice is not limited to text, and may be stored in the second DB as an image, a moving image, or the like.

(3rd DB)
In the third DB (third storage unit), the information of the material received by the receiving unit 501 (first receiving unit), which will be described later, is stored in association with the identification information (ID) of the customer CS. Here, the material is a material necessary for responding to the customer CS, and is, for example, a product specification. The material stored in the third DB can be read out as needed and displayed on the display device 300D of the OP terminal 3.

(4th DB)
Information for analyzing the operator OP and emotions is stored in the fourth DB. Specifically, in the 4th DB, a model that learns acoustic characteristics (characteristics such as voice pitch, speaking speed, voice volume, intonation, etc.) for each emotion (for example, "normal", "joy", "anger"). Is stored. In addition to the above "normal", "joy", and "anger", emotions such as "sadness" and "enjoyment" may be added, and further subdivided into "trust", "worry", "surprise", and "disgust". An emotion such as "interest" may be added and stored in the 4th DB in association with the acoustic characteristics. It is possible to respond to the subtleties of emotions and perform more accurate analysis. By storing the model in which the acoustic characteristics for each emotion are learned in this way in the fourth DB, it is possible to analyze what kind of emotion both the operator OP and the customer CS have by referring to the fourth DB. ..

(5th DB)
The fifth DB stores information for converting the voice into text from the waveform of the voice signal. Specifically, in the fifth DB, acoustic features such as voice signal waveforms and frequency characteristics and words are stored in association with each other. By referring to the fifth DB, it is possible to convert the voice into text from the acoustic features such as the waveform and frequency characteristics of the voice signal.

(6th DB)
In the sixth DB, a transition pattern of emotional sympathy between the operator OP and the customer CS (hereinafter, also simply referred to as an emotional transition pattern) is stored. For example, the closing rate is higher than the predetermined value (for example, the average value), the closing time is shorter than the predetermined time, the opening breakthrough rate is higher than the predetermined value (for example, the average value), and the like. The emotional sympathy is an index showing how much the emotions of the operator OP and the customer CS match (match rate), and the emotions of the operator OP and the customer CS (for example, emotions and sorrows) match. Empathy is determined by whether it is (synchronized). The higher the ratio of matching emotions of the operator OP and customer CS (for example, emotions), the higher the sympathy of emotions, and the lower the ratio of matching emotions of the operator OP and customer CS, the higher the sympathy of emotions. The degree is low. Here, in the sixth DB, the emotional transition pattern of the operator OP and the customer CS is stored according to the stage (talk phase, for example, sales, opening, offer, rebattle, closing, etc.). As a result, the evaluation unit 512, which will be described later, can evaluate the response to the customer CS by the operator OP based on different criteria depending on the stage, and can perform an appropriate evaluation with high accuracy.

The CPU 500C controls the server 5. The CPU 500C includes a ROM (Read Only Memory) and a RAM (Random Access Memory) (not shown).

(Functional configuration of server 5)
As shown in FIG. 4B, the server 5 includes a receiving unit 501 (first and second receiving units), a voice information acquisition unit 502, an acoustic feature extraction unit 503, an emotion analysis unit 504, a text generation unit 505, and a stage. Estimating unit 506, emotion pattern identification unit 507, talk script acquisition unit 508, conversation speed measurement unit 509, situation determination unit 510, advice acquisition unit 511, evaluation unit 512, storage device control unit 513, presentation unit 514, transmission unit 515, etc. Has the function of.

The receiving unit 501 includes information transmitted from the call terminal 2B (for example, the voice signal of the operator OP), information transmitted from the SV terminal 4, and information transmitted from the call terminal 2A via the PBX device 6 (for example,). Receive the voice signal of the customer CS, etc.). In addition, the receiving unit 501 receives (accepts) the information regarding the evaluation of the operator OP transmitted from the SV terminal 4. In addition, the receiving unit 501 receives (accepts) the storage operation of the material necessary for responding to the customer CS.

The voice information acquisition unit 502 acquires the voice signals of the customer CS and the operator OP transmitted from the call terminal 2A and the call terminal 2B.

The acoustic feature extraction unit 503 extracts the acoustic features of the operator OP and the customer CS from the voice signal acquired by the voice information acquisition unit 502. Here, the acoustic features include, for example, the pitch (frequency of voice), the loudness (amplitude of voice), and the waveform of voice. The acoustic feature extraction unit 503 extracts acoustic features from the voice signals of the operator OP and the customer CS while the call state between the call terminal 2A and the call terminal 2B continues.

The emotion analysis unit 504 analyzes the emotion of the operator OP from the acoustic features of the operator OP extracted by the acoustic feature extraction unit 503. Further, the emotion analysis unit 504 analyzes the emotion of the customer CS from the acoustic features of the customer CS extracted by the acoustic feature extraction unit 503. Specifically, the emotion analysis unit 504 analyzes the emotions of the operator OP and the customer CS by referring to the relationship between the acoustic features and the emotions stored in the fourth DB. In the present embodiment, the emotion analysis unit 504 calculates the emotions of the operator OP and the customer CS as emotion values (numerical values representing the estimated degree (magnitude) of emotions). That is, the time-series data of the emotion value becomes the emotion transition pattern. In other words, the emotion value is a part of the emotion transition pattern, and the value obtained by extracting the value of a certain temporary point from the emotion transition pattern is the emotion value. In addition, the emotion analysis unit 504 calculates the sympathy of both emotions from the analyzed emotions of the operator OP and the customer CS.

The text generation unit 505 refers to the fifth DB and converts the conversation content into text from the voice signals of the operator OP and the customer CS.

The stage estimation unit 506 determines which stage (talk phase, for example, sales, opening, offer, rebattle, closing) the operator OP responds to the customer CS based on the keywords included in the text data generated by the text generation unit 505. Etc.). For example, the stage estimation unit 506 performs morphological analysis on the text data generated by the text generation unit 505, and extracts keywords included in the text data. Then, based on which stage the extracted keywords are associated with, it is estimated at which stage (talk phase) the operator OP responds to the customer CS. In this way, when estimating which stage (talk phase) the operator OP responds to the customer CS based on which stage the extracted keyword is associated with, the keyword and the stage are associated with each other. The database may be provided, or the database may be referred to.

The emotion pattern specifying unit 507 specifies which emotion transition pattern is to be compared with one or more emotion transition patterns stored in the sixth DB. Specifically, the emotion pattern identification unit 507 stores the operator OP in the sixth DB according to the stage estimated by the stage estimation unit 506 among the one or more emotion transition patterns stored in the sixth DB. And the emotional transition pattern of the customer CS is specified as a comparison target.

In the present embodiment, the emotional transition pattern of the operator OP and the customer CS is stored in the sixth DB according to the stage estimated by the stage estimation unit 506, but the classification result (category) of the operator OP and the customer CS The sixth DB may store the emotional transition patterns of the operator OP and the customer CS, which are further subdivided such as the classification result (category), conversation date, conversation time zone, number of calls, progress status, and the like. In this case, the emotion pattern identification unit 507 includes the classification result (category) of the operator OP analyzing the emotion by the emotion analysis unit 504, the classification result (category) of the customer CS, the conversation date, the conversation time zone, the number of calls, and the progress. Identify transition patterns of emotions that match or resemble situations, etc.

When specifying the transition pattern of the most similar emotions, how to specify the transition pattern of the most similar emotions is arbitrary. For example, for each of the operator OP and the customer CS, cluster analysis is performed using each item such as category, conversation date, conversation time zone, number of calls, and progress status as data, and the similarity is calculated as the distance between each cluster. , The emotional transition pattern with the highest similarity value may be used as the emotional transition pattern for comparison.

Here, the fact that a certain item is included in a specific cluster is called "attribute matching". If the attributes do not match, the distance from the center of the cluster is calculated as the similarity (referred to here as "attribute similarity"). When each data (operator OP, customer CS category, conversation date, conversation time zone, number of calls, progress status, etc.) matches in all the clusters, it is considered as "match", and there are clusters that do not match even one. In this case, the cluster with the maximum attribute similarity of the clusters whose attributes do not match is regarded as "most similar". It should be noted that weighting may be set when determining the distance of each cluster to adjust the attribute similarity. In addition, the attribute class cluster analysis uses various known methods, a short-distance method, a group averaging method, and the like, and the distance to the cluster can be calculated using various known methods. For example, Euclidean distance, cosine distance, Hamming distance and the like can be used.

The talk script acquisition unit 508 refers to the first DB and acquires the talk script associated with the stage estimated by the stage estimation unit 506.

The conversation speed measurement unit 509 measures the conversation speed of the customer CS based on the voice signal of the customer CS acquired by the voice information acquisition unit 502.

The situation determination unit 510 determines what kind of state the situation between the operator OP and the customer CS is from the content of the conversation between the customer CS and the operator OP. More specifically, the situation determination unit 510 corresponds to one of the cases where the situation between the operator OP and the customer CS is stored in the second DB based on the conversation content converted into text by the text generation unit 505. Determine whether or not to do so. For example, the situation determination unit 510 performs morphological analysis on the text data generated by the text generation unit 505, and the number of times that the keyword included in the case stored in the second DB appears in the text data (hereinafter, the number of appearances). It may be determined whether or not the situation between the operator OP and the customer CS corresponds to any of the cases stored in the second DB. As a specific example, the situation determination unit 510 determines the situation between the operator OP and the customer CS according to the number of appearances of keywords (for example, refund, return, compensation, damage, etc.) included in the case stored in the second DB. It may be determined whether or not it corresponds to the case stored in the 2DB (for example, the customer CS speaks the complaint endlessly). Needless to say, the situation is not limited to the above specific example.

The advice acquisition unit 511 refers to the second DB and acquires advice to the operator OP according to the determination result in the situation determination unit 510. Specifically, the advice acquisition unit 511 acquires advice according to the case determined by the situation determination unit 510.

The evaluation unit 512 compares the emotional transition pattern specified by the emotion pattern specifying unit 507 with the emotional transition pattern of the operator OP and the customer CS analyzed by the emotion analysis unit 504, and transfers to the customer CS by the operator OP. Evaluate the response (good, bad, etc.). For example, in the evaluation unit 512, when the difference between the emotion transition pattern specified by the emotion pattern identification unit 507 and the emotion transition pattern of the operator OP and the customer CS analyzed by the emotion analysis unit 504 is less than a predetermined value. If the evaluation is "good (GOOD)" and the deviation is equal to or greater than a predetermined value, the evaluation may be "bad (BAD)". The evaluation based on the deviation of the transition pattern is just an example, and another evaluation method may be adopted as the evaluation method in the evaluation unit 512.

The storage device control unit 513 controls the storage device 500B. Specifically, the storage device control unit 513 controls the storage device 500B to write and read the information of the first DB to the sixth DB stored in the storage device 500B.

The presentation unit 514 presents the evaluation result of the evaluation unit 512. Specifically, the presentation unit 514 presents the following information to the OP terminal 3 and the SV terminal 4.
(1) Information on the evaluation result in the evaluation unit 512 (2) Character information in which at least one of the facial expression and the movement changes according to the evaluation result in the evaluation unit 512 and / or the information in the character image (3). ) Information on the evaluation (for example, advice and impressions) of the operator OP by the supervisor SV received by the receiving unit 501 (4) The stage of responding to the customer CS by the operator OP estimated by the stage estimation unit 506 (talk phase) : Sales, opening, offer, rebattle, closing, etc.) information (5) Talk script information acquired by the talk script acquisition unit 508 (6) Advice information acquired by the advice acquisition department 511

The transmission unit 515 transmits the information (1) to (6) above to the OP terminal 3 and / or the SV terminal 4 based on the instruction of the presentation unit 514. In the "presentation" of the present embodiment, not only the information of (1) to (6) above is displayed on the display devices 300D and 400D of the OP terminal 3 and / or the SV terminal 4, but also the above (1) It is also assumed that the information of (6) is transmitted from the server 5.

The PBX device 6 executes protocol conversion between the network 8 and the public network 7, call control for incoming and outgoing calls, and the like.

(Screen displayed on the display device 300D of OP terminal 3)
FIG. 5 is an example of a display screen displayed on the display device 300D of the OP terminal 3. As shown in FIG. 5, the display device 300D of the OP terminal 3 displays a character image 601 in which at least one of facial expressions and movements changes according to the evaluation result of the evaluation unit 512. In addition, instead of the character image 601, the character information may be displayed on the display device 300D of the OP terminal 3 according to the evaluation result by the evaluation unit 512, or both the character image 601 and the character information may be displayed on the display device. It may be displayed on 300D.

Further, the display device 300D of the OP terminal 3 displays the stage (talk phase) 602 estimated by the stage estimation unit 506. In the example shown in FIG. 5, the talk phase exists from the first stage to the fifth stage, and it is shown by hatching that the current stage estimated by the stage estimation unit 506 is the second stage. Here, for example, the first stage is "sales", the second stage is "opening", the third stage is "offer", the fourth stage is "rebattle", and the fifth stage is "closing". It is not limited to. Further, the talk phase in the present embodiment is not limited to five stages, and the number of stages of the talk phase is arbitrary. Further, in order to indicate which stage the current stage estimated by the stage estimation unit 506 is, a display mode other than hatching (for example, a difference in background color) may be used.

Further, on the display device 300D of the OP terminal 3, the talk script acquired by the talk script acquisition unit 508 is displayed in the display frame 603. The talk script displayed in the display frame 603 is displayed in a mode in which the characters in the reading portion are different (for example, the color of the characters changes) according to the conversation speed measured by the conversation speed measuring unit 509. If the read part can be understood, another display mode may be used. For example, the read part may be understood by scrolling the talk script left / right or up / down. Further, the talk script may be combined with a mode in which the read portion can be understood by scrolling left and right or up and down and a mode in which the characters in the read portion are different (for example, the color of the characters changes).

Further, the display device 300D of the OP terminal 3 displays information 604 regarding the evaluation of the operator OP transmitted from the SV terminal 4 and received by the reception unit 501, advice 605 acquired by the advice acquisition unit 511, and the like. The information 604 regarding the evaluation of the operator OP transmitted from the SV terminal 4 and received by the receiving unit 501 may be a comment (character information) input by the supervisor SV by operating the input device 400C of the SV terminal 4. It may be an image showing the evaluation of the operator OP by the supervisor SV (for example, an image of a trophy), or may be a combination of the above comment (text information) and the above image. Further, at least one of the comment (character information) and the image may be configured to be displayable on the display device 300D of the OP terminal 3.

The display screen shown in FIG. 5 is an example of a screen displayed on the display device 300D of the OP terminal 3, and the above-mentioned items (character image 601, stage (talk phase) 602, talk script display frame 603, and so on. It is arbitrary at which position on the screen of the display device 300D the information 604 regarding the evaluation of the operator OP, the advice 605, and the information 606) regarding the response to the customer CS are presented.

(Processing by information processing system)
6 to 11 are flowcharts showing an example of processing by the information processing system 1. Hereinafter, processing by the information processing system 1 will be described with reference to FIGS. 6 to 11.

(Main processing)
FIG. 6 is a diagram showing a main process by the information processing system 1. Hereinafter, the main processing of the information processing system 1 will be described with reference to FIG.

(Step S101)
The voice information acquisition unit 502 of the server 5 acquires the voice signals of the customer CS and the operator OP transmitted from the call terminal 2A and the call terminal 2B.

(Step S102)
The acoustic feature extraction unit 503 of the server 5 extracts the acoustic features of the operator OP and the customer CS from the voice signal acquired by the voice information acquisition unit 502. The acoustic feature extraction unit 503 extracts acoustic features from the voice signals of the operator OP and the customer CS while the call between the call terminal 2A and the call terminal 2B continues.

(Step S103)
The emotion analysis unit 504 of the server 5 executes the emotion analysis process. The details of the emotion analysis process will be described later.

(Step S104)
The server 5 executes the evaluation presentation process. The details of the evaluation presentation process will be described later.

(Step S105)
The server 5 executes the talk script presentation process. The details of the talk script presentation process will be described later.

(Step S106)
The server 5 executes the advice presentation process. The details of the advice presentation process will be described later.

(Step S107)
Specifically, the server 5 determines whether or not the call between the call terminal 2A and the call terminal 2B is continuing. When the conversation between the operator OP and the customer CS continues (YES in step S107), the server 5 continues the processing of steps S101 to S106. In other words, the server 5 continues the processing of steps S101 to S106 while the call between the calling terminal 2A and the calling terminal 2B continues. If the conversation between the operator OP and the customer CS is not continuing (NO in step S107), the server 5 ends the evaluation process. The end of the evaluation process may be determined based on whether or not the call with the call terminal 2A and the call terminal 2B is blocked.

The storage device control unit 513 of the server 5 is analyzed by various information acquired / extracted by the server 5, for example, the voice signals of the operator OP and the customer CS acquired by the voice information acquisition unit 502, and the emotion analysis unit 504. The analysis result of the emotions of the operator OP and the customer CS, the text data of the conversation between the operator OP and the customer CS generated by the text generation unit 505, etc. are associated with the identifier (ID) of the operator OP and the customer CS and stored in the storage device 500B. Store. In addition to the IDs of the operator OP and the customer CS, an identifier (conversation ID) may be associated with the text data and stored in the storage device 500B so that individual conversations can be identified. ..

(Emotion analysis processing)
FIG. 7 is a diagram showing an emotion analysis process by the information processing system 1. Hereinafter, the emotion analysis process by the information processing system 1 will be described with reference to FIG. 7.

(Step S201)
The emotion analysis unit 504 reads the acoustic characteristics of the first model (for example, a “normal” learning model) among the trained models stored in the fourth DB.

(Step S202)
The emotion analysis unit 504 compares the acoustic characteristics of the operator OP and the customer CS extracted by the acoustic feature extraction unit 503 with the acoustic characteristics of the read first model, and calculates the similarity. The degree of similarity can be calculated by using various known methods. For example, using the Euclidean distance, Pearson's product moment correlation coefficient, Jaccard coefficient, etc., the similarity between the acoustic characteristics of the operator OP and customer CS extracted by the acoustic feature extraction unit 503 and the acoustic characteristics of the first model. Can be calculated.

(Step S203)
The emotion analysis unit 504 reads the acoustic characteristics of the second model (for example, the learning model of "joy") among the trained models stored in the fourth DB.

(Step S204)
The emotion analysis unit 504 compares the acoustic characteristics of the operator OP and the customer CS extracted by the acoustic feature extraction unit 503 with the acoustic characteristics of the read second model, and calculates the similarity. The same method as in step S202 can be used for calculating the degree of similarity.

(Step S205)
The emotion analysis unit 504 reads the acoustic characteristics of the third model (for example, the learning model of “anger”) among the trained models stored in the fourth DB.

(Step S206)
The emotion analysis unit 504 compares the acoustic characteristics of the operator OP and the customer CS extracted by the acoustic feature extraction unit 503 with the acoustic characteristics of the read third model, and calculates the similarity. The same method as in step S202 can be used for calculating the degree of similarity.

(Step S207)
The emotion analysis unit 504 adjusts the similarity by weighting the similarity as compared with the first to third models.

(Step S208)
The emotion analysis unit 504 estimates the emotions of the operator OP and the customer CS based on the degree of similarity with each of the adjusted first to third models. More specifically, the emotion analysis unit 504 estimates the emotions of the operator OP and the customer CS, respectively, depending on how close the acoustic features of the operator OP and the customer CS are to which of the first to third models. The same method as in step S202 can be used to calculate how close the acoustic features of the operator OP and the customer CS are to which of the first to third models.

(Step S209)
The emotion analysis unit 504 calculates the sympathy of both emotions from the analyzed emotions of the operator OP and the customer CS, in other words, the matching rate (sympathy) of the emotions of the operator OP and the customer CS. Here, the sympathy is time-series data, and the emotion analysis unit 504 calculates the emotional match rate (empathy) based on the characteristic pattern of the emotion value for each utterance of the operator OP and the customer CS. The emotional concordance rate (empathy) is mainly based on the timing at which emotional values other than "normal" of the customer CS occur, the amount of change in emotional value, etc., and the utterance timing of the operator OP in response to the change in the characteristic amount. The emotion value and the amount of change in the emotion value are calculated as parameters. The emotional concordance rate (empathy) can be calculated by combining various known methods. For example, machine learning may be used to extract the characteristics of time series data of emotional concordance rate (empathy).

In the above emotion analysis process (FIG. 7), the emotions of the operator OP and the customer CS are estimated based on the similarity with the three models of the first to third models, but the trained model is It is not limited to only three (for example, "normal", "joy", and "anger"), but may be arbitrary, and similarity with the learned model such as "sadness" and "enjoyment" may be added, or further subdivided. Similarities with trained models such as "trust", "worry", "surprise", "disgust", and "interest" may be added.

(Evaluation presentation processing)
FIG. 8 is a diagram showing an evaluation presentation process by the information processing system 1. Hereinafter, the evaluation presentation process by the information processing system 1 will be described with reference to FIG.

(Step S301)
The stage estimation unit 506 of the server 5 estimates at which stage the operator OP responds to the customer CS based on the keywords included in the text data generated by the text generation unit 505.

(Step S302)
The emotion pattern specifying unit 507 of the server 5 specifies which emotion transition pattern is to be compared with one or more emotion transition patterns stored in the sixth DB. Specifically, the emotion pattern identification unit 507 stores the operator OP in the sixth DB according to the stage estimated by the stage estimation unit 506 among the one or more emotion transition patterns stored in the sixth DB. And the emotional transition pattern of the customer CS is specified as a comparison target.

(Step S303)
The evaluation unit 512 of the server 5 compares the emotion transition pattern specified by the emotion pattern identification unit 507 with the emotion transition pattern of the operator OP and the customer CS analyzed by the emotion analysis unit 504, and uses the operator OP. Evaluate the response to customer CS. As described above, in the evaluation unit 512, the discrepancy between the emotion transition pattern specified by the emotion pattern specifying unit 507 and the emotion transition pattern of the operator OP and the customer CS analyzed by the emotion analysis unit 504 is less than a predetermined value. If this is the case, the evaluation may be "good (GOOD)", and if the deviation is greater than or equal to a predetermined value, the evaluation may be "bad (BAD)" or another evaluation method may be adopted.

(Step S304)
The presentation unit 514 of the server 5 presents the evaluation result of the evaluation unit 512. Specifically, the presentation unit 514 instructs the transmission unit 515 to transmit the information of the evaluation result in the evaluation unit 512 to the OP terminal 3 of the operator OP who performed the evaluation. Further, the presentation unit 514 instructs the transmission unit 515 to transmit the information of the evaluation result in the evaluation unit 512 to the SV terminal 4 of the supervisor SV who is the manager of the operator OP who performed the evaluation. The transmission unit 515 transmits the information of the evaluation result in the evaluation unit 512 to the OP terminal 3 and the SV terminal 4 based on the instruction of the presentation unit 514. The information transmitted from the transmission unit 515 is received by the reception unit 301 of the OP terminal 3 and the reception unit 401 of the SV terminal 4, respectively, and is received by the display device control unit 304 of the OP terminal 3 and the display device control unit 404 of the SV terminal 4. It is presented to the display device 300D and the display device 400D, respectively. As described with reference to FIG. 5, the display device 300D is presented as a character image in which at least one of the facial expression and the motion changes according to the evaluation result of the evaluation unit 512 and / or the expression result. ..

(Talk script presentation process)
FIG. 9 is a diagram showing a talk script presentation process by the information processing system 1. Hereinafter, the talk script presentation process by the information processing system 1 will be described with reference to FIG.

(Step S401)
The stage estimation unit 506 of the server 5 estimates at which stage the operator OP responds to the customer CS based on the keywords included in the text data generated by the text generation unit 505.

(Step S402)
The talk script acquisition unit 508 of the server 5 refers to the first DB and acquires the talk script associated with the topic estimated by the stage estimation unit 506.

(Step S403)
The conversation speed measurement unit 509 of the server 5 measures the conversation speed of the customer CS based on the voice signal of the customer CS acquired by the voice information acquisition unit 502.

(Step S404)
The presentation unit 514 of the server 5 presents the talk script information acquired by the talk script acquisition unit 508. At this time, the presentation unit 514 displays the characters in the reading portion in a different manner (for example, the color of the characters changes) according to the conversation speed measured by the conversation speed measurement unit 509. Present the information of the talk script obtained in. As described with reference to FIG. 5, another display mode may be used as long as the read portion can be understood. For example, the read portion may be understood by scrolling the talk script left / right or up / down. Further, the talk script may be combined with a mode in which the read portion can be understood by scrolling left and right or up and down and a mode in which the characters in the read portion are different (for example, the color of the characters changes). Specifically, the presentation unit 514 instructs the transmission unit 515 to transmit the talk script information acquired by the talk script acquisition unit 508 to the OP terminal 3 of the operator OP. The transmission unit 515 transmits the information of the talk script acquired by the talk script acquisition unit 508 to the OP terminal 3 based on the instruction of the presentation unit 514. The information transmitted from the transmission unit 515 is received by the reception unit 301 of the OP terminal 3 and then displayed on the display device control unit 304 of the OP terminal 3.

(Advice presentation processing)
FIG. 10 is a diagram showing an advice presentation process by the information processing system 1. Hereinafter, the advice presentation process by the information processing system 1 will be described with reference to FIG.

(Step S501)
The situation determination unit 510 determines whether or not the situation between the operator OP and the customer CS corresponds to any of the cases stored in the second DB from the contents of the conversation between the customer CS and the operator OP. When there is a corresponding case (YES), the situation determination unit 510 shifts to the process of step S502. Further, the situation determination unit 510 ends the advice presentation process when there is no corresponding case (NO).

(Step S502)
The advice acquisition unit 511 refers to the second DB and acquires advice to the operator OP according to the determination result in the situation determination unit 510. Specifically, the advice acquisition unit 511 acquires advice according to the case determined by the situation determination unit 510.

(Step S503)
The presentation unit 514 of the server 5 presents the advice acquired by the advice acquisition unit 511. Specifically, the presentation unit 514 instructs the transmission unit 515 to transmit the advice information acquired by the advice acquisition unit 511 to the OP terminal 3 of the operator OP. The transmission unit 515 transmits the advice information acquired by the advice acquisition unit 511 to the OP terminal 3 based on the instruction of the presentation unit 514. The information transmitted from the transmission unit 515 is received by the reception unit 301 of the OP terminal 3 and then displayed on the display device control unit 304 of the OP terminal 3.

(SV presentation processing)
FIG. 11 is a diagram showing an SV presentation process for presenting an evaluation from the supervisor SV by the information processing system 1. Hereinafter, the SV presentation process by the information processing system 1 will be described with reference to FIG.

(Step S601)
The input receiving unit 403 of the SV terminal 4 receives information on the evaluation (for example, advice, impression, etc.) of the operator OP input by the supervisor SV by operating the input device 400C.

(Step S602)
The transmission unit 405 of the SV terminal 4 transmits the information received by the input reception unit 403 to the server 5.

(Step S603)
The receiving unit 501 of the server 5 receives the information transmitted from the SV terminal 4. The presentation unit 514 of the server 5 presents information regarding the evaluation of the operator OP received by the reception unit 501. Specifically, the presentation unit 514 instructs the transmission unit 515 to transmit the information regarding the evaluation of the operator OP received by the reception unit 501 to the OP terminal 3 of the operator OP. The transmission unit 515 transmits the information regarding the evaluation of the operator OP received by the reception unit 501 to the OP terminal 3 based on the instruction of the presentation unit 514. The information transmitted from the transmission unit 515 is received by the reception unit 301 of the OP terminal 3 and then displayed on the display device control unit 304 of the OP terminal 3. It should be noted that the configuration may be such that information regarding the evaluation of the operator OP can be directly transmitted from the SV terminal 4 to the OP terminal 3 via the network 8 without going through the server 5.

As described above, the server 5 of the information processing system 1 according to the present embodiment acquires the voice information of the customer CS (first speaker) and the operator OP (second speaker) who talks with the customer CS. The emotion analysis unit 504 that analyzes the emotions of the customer CS and the operator OP from the voice information acquired by the unit 502 and the voice information acquisition unit 502, and the operator OP to the customer CS according to the analysis result in the emotion analysis unit 504. It includes an evaluation unit 512 that evaluates correspondence and a presentation unit 514 that presents the evaluation results of the evaluation unit 512.
Therefore, the evaluation of the response to the customer CS by the operator OP can be fed back to the operator OP itself and the supervisor SV, and the customer CS may be in a complaint state of being dissatisfied with the response of the operator OP. It can be suppressed and is excellent in convenience. Further, in the present embodiment, since the emotions of the customer CS and the operator OP are analyzed in real time, feedback can be obtained immediately, and the response of the operator OP can be immediately improved, which is convenient. Excellent.

Further, the presentation unit 514 of the server 5 presents a character image in which at least one of the facial expression and the motion changes according to the evaluation result and / or the character information according to the evaluation result.
Since the character image in which at least one of the character information, the facial expression, and the movement changes according to the evaluation result is presented to the OP terminal, the operator OP can obtain feedback even when talking with the customer CS, which is excellent in convenience. .. Further, when presenting a character image in which at least one of facial expressions and movements changes, the form is indirectly pointed out, so that the operator OP can easily accept the evaluation result and reflect it in his / her own response.

Further, the presentation unit 514 of the server 5 presents the evaluation result of the evaluation unit 512 to the SV terminal 4 of the supervisor SV (administrator) who is the administrator of the operator OP (second speaker). Since the evaluation of the operator OP is also presented to the SV terminal 4 of the supervisor SV, the supervisor SV can know the correspondence status of the operator OP to the customer CS, which is excellent in convenience.

Further, the server 5 includes a receiving unit 501 (second reception unit) that receives information related to the evaluation of the operator OP transmitted from the SV terminal 4, and the presentation unit 514 is received by the receiving unit 501 (second reception unit). Information on the evaluation of the operator OP is presented to the OP terminal 3 of the operator OP.
The operator OP can receive feedback from the supervisor SV who has confirmed the evaluation of the response to the customer CS, and is excellent in convenience. Further, the supervisor SV can also send feedback from the SV terminal 4 to the operator OP, which is excellent in convenience.

Further, the presentation unit of the server 5 presents a stage (talk phase, for example, sales, opening, offer, rebattle, closing, etc.) of the operator OP in response to the customer CS. Therefore, the operator OP can grasp at which stage the current conversation is in the product explanation and the contract content explanation, which is excellent in convenience.

Further, the evaluation unit 512 of the server 5 evaluates the response to the customer CS by the operator OP according to different criteria depending on the stage.
In this way, by evaluating the response of the operator OP to the customer CS based on different criteria depending on the stage, more accurate evaluation can be performed, which is excellent in convenience.

Further, the server 5 includes a stage estimation unit 506 that estimates the stage from the conversation contents of the customer CS and the operator OP. Then, the presentation unit 514 of the server 5 presents the stage estimated by the stage estimation unit 506.
Therefore, the operator OP does not have to grasp the stage of the conversation with the customer CS by himself / herself, and can concentrate on the conversation with the customer CS, which is excellent in convenience. In addition, since the operator OP can concentrate on the conversation with the customer CS, it can be expected that the results of outbound sales and inbound sales will improve.

Further, the server 5 refers to the first DB (first storage unit) in which the stage and the talk script related to the stage are stored in association with each other, and acquires the talk script associated with the stage estimated by the stage estimation unit 506. It includes a talk script acquisition unit 508. Then, the presentation unit 514 of the server 5 presents the talk script acquired by the talk script acquisition unit 508.
In this way, the talk script along the stage (talk phase) is automatically acquired and presented to the operator OP, so it can be expected that the burden of conversation with the customer CS of the operator OP will be significantly reduced, which is convenient. Excellent in sex.

Further, the server 5 includes a conversation speed measuring unit 509 that measures the conversation speed of the customer CS. Then, the presentation unit 514 of the server 5 presents the talk script according to the conversation speed measured by the conversation speed measurement unit 509.
In this way, since the talk script is presented according to the conversation speed of the customer CS, the operator OP does not hesitate about how to read the talk script, and the burden of conversation with the customer CS of the operator OP is burdened. It can be expected to be significantly reduced and is excellent in convenience.

Further, the server 5 has a situation determination unit 510 that determines the situation between the operator OP and the customer CS from the conversation content between the customer CS and the operator OP, and a second DB (second DB) in which information regarding advice to the operator OP is stored. It is provided with an advice acquisition unit 511 that acquires advice to the operator OP according to the determination result in the situation determination unit 510 with reference to the storage unit). Then, the presentation unit 514 of the server 5 presents the advice acquired by the advice acquisition unit 511.
Therefore, the situation is determined between the operator OP and the customer CS from the conversation content between the customer CS and the operator OP, and advice to the operator OP is obtained according to the determination result in the situation determination unit 510. Therefore, the operator OP gives advice. By following the above, it can be expected that the results of outbound sales and inbound sales will improve.

Further, the server 5 receives the materials received by the reception unit 501 (first reception unit) and the reception unit 501 (first reception unit) that receive the storage of the materials necessary for responding to the customer CS. It includes a storage device control unit 513 (memory control unit) that is associated with the identification information and stored in the third DB (third storage unit).
Therefore, the operator OP can confirm the material previously presented to the customer CS, which is excellent in convenience. In addition, since the materials can be confirmed immediately, it can be expected that the satisfaction level of the customer CS will be improved.

[Modification 1 of the embodiment]
FIG. 12 is a diagram showing an example of the functional configuration of the server 5 in the modified example 1 of the embodiment. Further, FIG. 13 is a flowchart showing an example of correspondence information presentation processing by the information processing system 1 according to the first modification of the embodiment. Hereinafter, the information processing system 1 according to the modification 1 of the embodiment will be described with reference to FIGS. 12 and 13, but the same as the information processing system 1 according to the embodiment described with reference to FIGS. 1 to 11. With respect to the configurations of the above, different configurations will be described with the same reference numerals, and duplicate description will be omitted.

In the first modification of the embodiment, the storage device 500B of the server 5 has a seventh DB (fourth storage unit). In the seventh DB, the emotion of the customer CS and the information regarding the response to the customer CS are stored in association with each other. The emotion is the emotion of the customer CS analyzed by the emotion analysis unit 504. In addition, the information on the response is, for example, a response method according to the emotional state of the customer CS, such as speaking slowly when the emotion of the customer CS is angry.

As shown in FIG. 12, the server 5 according to the modification 1 includes a receiving unit 501 (first and second receiving units), a voice information acquisition unit 502, an acoustic feature extraction unit 503, an emotion analysis unit 504, and a text generation unit 505. , Stage estimation unit 506, emotion pattern identification unit 507, talk script acquisition unit 508, conversation speed measurement unit 509, situation determination unit 510, advice acquisition unit 511, evaluation unit 512, storage device control unit 513, presentation unit 514, transmission unit. It has functions such as 515 and a corresponding information acquisition unit 516.

The correspondence information acquisition unit 517 refers to the 7th DB, and acquires the information regarding the correspondence to the customer CS according to the analysis result of the emotion of the customer CS by the emotion analysis unit 504. In addition, the information regarding the correspondence to the customer CS acquired by the correspondence information acquisition unit 517 is presented to the OP terminal 3 by the presentation unit 514.

(Correspondence information presentation processing)
FIG. 13 is a diagram showing correspondence information presentation processing by the information processing system 1 according to the first modification. Hereinafter, the corresponding information presentation process by the information processing system 1 will be described with reference to FIG.

(Step S701)
The correspondence information acquisition unit 517 of the server 5 (information processing device) according to the first modification refers to the 7th DB and provides information on the correspondence to the customer CS according to the analysis result of the emotion of the customer CS by the emotion analysis unit 504. get. Since the method of analyzing the emotions of the customer CS by the emotion analysis unit 504 has been described with reference to FIG. 7, duplicate explanations will be omitted.

(Step S702)
The presentation unit 514 of the server 5 according to the first modification presents the information regarding the correspondence to the customer CS acquired by the correspondence information acquisition unit 517. Specifically, the presentation unit 514 instructs the transmission unit 515 to transmit the information regarding the correspondence to the customer CS acquired by the correspondence information acquisition unit 517 to the OP terminal 3 of the operator OP. Based on the instruction of the presentation unit 514, the transmission unit 515 transmits the information regarding the correspondence to the customer CS acquired by the correspondence information acquisition unit 517 to the OP terminal 3. The information transmitted from the transmission unit 515 is received by the reception unit 301 of the OP terminal 3 and then displayed on the display device control unit 304 of the OP terminal 3.

As described above, the server 5 according to the first modification of the embodiment refers to the seventh DB in which the emotion of the customer CS and the information related to the correspondence to the customer CS are stored in association with each other, and the analysis result by the emotion analysis unit 504. It is provided with a correspondence information acquisition unit 517 that acquires information regarding correspondence to the customer CS according to the situation. Then, the presentation unit 514 of the server 5 presents the information regarding the correspondence to the customer CS acquired by the correspondence information acquisition unit 517. Therefore, the response can be changed immediately according to the emotion of the customer CS, which is excellent in convenience.

[Modification 2 of the embodiment]
In the above embodiment and the first modification, the conversation speed measurement unit 509 of the server 5 measures the conversation speed of the customer CS, and the presentation unit 514 of the server 5 is the talk script acquired by the talk script acquisition unit 508. The information is displayed on the display device 300D of the OP terminal 3 in a manner in which the characters in the reading portion are different (for example, the color of the characters changes) according to the conversation speed of the customer CS measured by the conversation speed measuring unit 509. It is presented to.

However, the conversation speed measurement unit 509 of the server 5 measures the conversation speed of the operator OP based on the voice signal of the operator OP acquired by the voice information acquisition unit 502, and the presentation unit 514 of the server 5 acquires the talk script. The OP terminal 3 uses the information of the talk script acquired by the unit 508 in a mode in which the characters in the read portion differ according to the conversation speed of the operator OP measured by the conversation speed measurement unit 509 (for example, the color of the characters changes). It may be configured to be presented to the display device 300D of. In this way, by presenting the talk script according to the conversation speed of the operator OP measured by the conversation speed measurement unit 509, the operator OP does not have to forcibly increase or decrease the reading speed and feels natural. You can read the talk script at, and you can expect to make a good impression on the customer CS.

Further, the conversation speed measurement unit 509 of the server 5 measures the conversation speed of the customer CS and the operator OP based on the voice signals of the customer CS and the operator OP acquired by the voice information acquisition unit 502, and the presentation unit of the server 5 In 514, the characters of the read portion of the talk script information acquired by the talk script acquisition unit 508 are different depending on the conversation speed of the customer CS and the operator OP measured by the conversation speed measurement unit 509 (for example, of characters). (The color changes) may be configured to be presented to the display device 300D of the OP terminal 3. In this case, the average value of the conversation speed of the customer CS and the conversation speed of the operator OP may be used as the conversation speed, or the conversation speed of the customer CS and the conversation speed of the operator OP are weighted to calculate the conversation speed. You may try to do it. By considering the conversation speed of both the customer CS and the operator OP, it can be expected that the talk script can be read aloud more effectively.

As described with reference to FIG. 5, other display modes may be used as long as the read aloud portion can be understood. For example, the read aloud portion may be understood by scrolling the talk script left / right or up / down. The characters in the places may be combined with different modes (for example, the color of the characters changes).

[Modification 3 of the embodiment]
In the above-described embodiment and the first and second modifications, the transition pattern of the emotional sympathy of the operator OP and the customer CS is stored in the sixth DB for each stage, but the transition pattern of the emotion of the operator OP and the customer CS is stored. May be stored in the 6th DB for each combination of customer CS category, operator OP category, conversation date, conversation time zone, number of calls, and progress status. In this case, the emotion transition patterns of the operator OP and the customer CS analyzed by the emotion analysis unit 504 are stored in the sixth DB specified by the emotion pattern specifying unit 507, and the emotions of the operator OP and the customer CS are stored. The advice acquisition unit 511 may extract the advice in comparison with the transition pattern of, and the presentation unit 514 may present the advice.

[Modification 4 of the embodiment]
Further, in the above-described embodiments and modifications 1 to 3, the emotional transition patterns of the operator OP and the customer CS are stored in the sixth DB in advance, but the operator OP and the customer analyzed by the emotion analysis unit 504 are stored in the server 5. A learning unit that learns the emotional transition pattern of CS for each operator OP is provided, and the emotional transition pattern of the operator OP and the customer CS stored in the 6th DB is updated by the learning unit according to the learning result. Good. By doing so, the transition of the emotions of the operator OP and the customer CS analyzed by the emotion analysis unit 504 is uniformly compared with the transition pattern of the emotions of the operator OP and the customer CS stored in the 6th DB. Not only can the advice be presented, but the advice can be presented according to the individuality and characteristics of each operator OP (for example, sales style), and sales can be supported more effectively.

[Modification 5 of the embodiment]
Further, in the above-described embodiment and the modified examples 1 to 4, the operator OP calls from the call terminal 2B to the call terminal 2A of the customer CS, but the server 5 and / or the OP terminal 3 makes a call to the customer CS. The configuration for calling the terminal 2A is not excluded, and the configuration may be such that the server 5 and / or the OP terminal 3 can call the call terminal 2A of the customer CS.

[Modification 6 of the embodiment]
Further, in the above-described embodiments and modifications 1 to 5, as shown in FIG. 1, in the information processing system 1, the PBX device 6 is connected to the call terminal 2B of the operator OP (first speaker) via the network 8. The OP terminal 3, the SV terminal 4, and the server 5 (information processing device) are communicably connected to the network 8 so as to make a call, but the network 8 may be configured as follows.
(1) The PBX device 6 is connected to the public network 7, the network 8 is connected to the public network 7 without going through the PBX device 6, and the call terminal 2B, the server 5 (information processing device), and the OP terminal are connected to the network 8. A configuration in which 3 and the SV terminal 4 are connected so as to be able to communicate.
(2) The PBX device 6 and the server 5 (information processing device) are connected to the public network 7, the network 8 is connected to the public network 7 without going through the PBX device 6, and the call terminal 2B and OP terminal are connected to the network 8. A configuration in which 3 and the SV terminal 4 are connected so as to be able to communicate.

The emotion analysis unit 504 of the server 5 may analyze the emotions of both speakers in consideration of the utterance contents of the operator OP and the customer CS (both speakers). In this way, by analyzing the emotions in consideration of the utterance content, it is possible to accurately analyze the emotions of the operator OP and the customer CS (both speakers). As a result, more effective sales support can be provided.

1 Information processing system 2A Call terminal (user)
2B call terminal (operator)
3 Operator terminal (OP terminal)
300A communication IF
300B storage device 300C input device 300D display device 300E CPU
301 Receiver (Reception)
302 Storage device control unit 303 Input reception unit 304 Display device control unit 305 Transmission unit 4 Supervisor terminal (SV terminal)
400A communication IF
400B Storage device 400C Input device 400D Display device 400E CPU
401 Receiver (Reception)
402 Storage device control unit 403 Input reception unit 404 Display device control unit 405 Transmission unit 5 Server (information processing device)
500A communication IF
500B storage device 500C CPU
501 Reception unit 502 Voice information acquisition unit 503 Acoustic feature extraction unit 504 Emotion analysis unit 505 Text generation unit 506 Stage estimation unit 507 Emotion pattern identification unit 508 Talk script acquisition unit 509 Conversation speed measurement unit 510 Situation judgment unit 512 Advice acquisition unit 513 Evaluation Unit 514 Storage device Control unit 515 Presentation unit 516 Transmission unit 511 Corresponding information acquisition unit 6 PBX device 7 Public network 8 Network CS Customer OP Operator SV Supervisor

Claims (14)

A voice information acquisition unit that acquires voice information of the first speaker and the second speaker who talks with the first speaker, and
An emotion analysis unit that analyzes the emotions of the first speaker and the second speaker from the voice information acquired by the voice information acquisition unit.
An evaluation unit that evaluates the response of the second speaker to the first speaker according to the analysis result of the emotion analysis unit.
A presentation unit that presents the evaluation results of the evaluation unit,
An information processing device characterized by being equipped with. The presentation unit
The information processing apparatus according to claim 1, wherein character information and / or a character image in which at least one of facial expressions and movements changes according to the evaluation result is presented. The presentation unit
The information processing apparatus according to claim 1 or 2, wherein the evaluation result in the evaluation unit is presented to the terminal of the administrator who manages the second speaker. It is provided with a first reception unit that receives information regarding the evaluation of the second speaker transmitted from the terminal of the administrator.
The presentation unit
The information processing apparatus according to claim 3, further comprising presenting information regarding the evaluation of the second speaker received by the first reception unit to the terminal of the second speaker. The presentation unit
The information processing apparatus according to any one of claims 1 to 4, wherein the stage of dealing with the first speaker by the second speaker is presented. The evaluation unit
The information processing apparatus according to claim 5, wherein the response of the second speaker to the first speaker is evaluated according to different criteria according to the stage. A stage estimation unit that estimates the stage from the conversation contents of the first speaker and the second speaker is provided.
The presentation unit
The information processing apparatus according to claim 5 or 6, wherein the stage estimated by the stage estimation unit is presented. A talk script acquisition unit that refers to a first storage unit in which the stage and a talk script related to the stage are stored in association with each other and acquires a talk script associated with the stage estimated by the stage estimation unit is provided. ,
The presentation unit
The information processing apparatus according to claim 7, wherein the talk script acquired by the talk script acquisition unit is presented. A conversation speed measuring unit for measuring the conversation speed of at least one of the first speaker and the second speaker is provided.
The presentation unit
The information processing device according to claim 8, wherein the talk script is presented according to the conversation speed measured by the conversation speed measuring unit. A situation determination unit that determines the situation between the second speaker and the first speaker from the conversation content between the first speaker and the second speaker.
With reference to the second storage unit in which the information regarding the advice to the second speaker is stored, the advice acquisition unit that acquires the advice to the second speaker according to the determination result in the situation determination unit. Prepare,
The presentation unit
The information processing device according to any one of claims 1 to 9, wherein the advice obtained by the advice acquisition unit is presented. The second reception section that accepts the storage of materials necessary for responding to the first speaker,
A storage control unit that stores the material received by the second reception unit in the third storage unit in association with the identification information of the first speaker.
The information processing apparatus according to any one of claims 1 to 10, wherein the information processing device is provided. The first speaker refers to the fourth storage unit in which the emotion of the first speaker and the information related to the correspondence to the first speaker are stored in association with each other, and according to the analysis result in the emotion analysis unit. It is equipped with a correspondence information acquisition department, which acquires information on correspondence to
The presentation unit
The information processing device according to any one of claims 1 to 11, wherein the information processing device according to claim 1 is provided with information relating to the correspondence to the first speaker acquired by the correspondence information acquisition unit. The process in which the voice information acquisition unit acquires the voice information of the first speaker and the second speaker who talks with the first speaker,
A step in which the emotion analysis unit analyzes the emotions of the first speaker and the second speaker from the voice information acquired by the voice information acquisition unit.
A step in which the evaluation unit evaluates the response of the second speaker to the first speaker according to the analysis result of the emotion analysis unit.
The process in which the presenting unit presents the evaluation result in the evaluation unit,
An information processing method characterized by having. Computer,
A voice information acquisition unit that acquires voice information of a first speaker and a second speaker who talks with the first speaker.
An emotion analysis unit that analyzes the emotions of the first speaker and the second speaker from the voice information acquired by the voice information acquisition unit.
An evaluation unit that evaluates the response of the second speaker to the first speaker according to the analysis result of the emotion analysis unit.
A presentation unit that presents the evaluation results of the evaluation unit,
An information processing program characterized by functioning as.

Images