JP4438014B1 - Harmful customer detection system, method thereof and harmful customer detection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect in real time whether or not a speaker in a call corresponds to each of all harmful customers registered in advance, and warn a telephone representative.
SOLUTION: With reference to a voiceprint database, relative distance is calculated only between voiceprint information of a source speaker and voiceprint information of a reference person of each group registered in the voiceprint database, and the relative distance is calculated Between the group selection unit which selects the group to which the reference person having the smallest voiceprint information belongs, the voiceprint information of the originating speaker, and the voiceprint information of the registered speaker other than the reference person belonging to the selected group The relative distance is calculated, and the calculation of the relative distance is repeated for all registered speakers other than the reference person, and voiceprint information having the smallest relative distance in the selected group is included, and the relative distance is the first And a registered speaker selection unit for selecting a registered speaker within a second threshold smaller than the threshold.
[Selected figure] Figure 1

Description

  The present invention relates to a harmful customer detection system, a method thereof and a harmful customer detection program. More specifically, for example, in a Customer Relationship Management (CRM) system that records and manages calls made between a customer's phone and a person-in-charge's phone, a harmful customer who interferes with smooth telephone handling operations is For example, the present invention relates to a technology for automatically detecting a person in charge of a response by detecting the speaker identification technology.

  There have been proposed various techniques for recording and managing voice calls made between a customer and a carrier at the carrier side.

  For example, Patent Document 1 discloses a centralized call recording system for digitizing, recording and searching an operator's call contents in a call center which is a telephone answering department from a customer.

  In general, a switchboard (PBX) where calls and calls from a Public Switched Telephone Network (PSTN) are concentrated is installed in a premises such as a call center operated by a business operator, and a voice call is made by this switch. Distributed to multiple landlines in call center premises. Therefore, if a call recording server branched from the exchange is provided, the call can be recorded and stored. On the operator side, a terminal device such as a PC may be provided together with a telephone answering extension telephone, and this operator terminal device has a function of searching customer information using a customer name told by a speaker as a key, and the customer A function may be provided to display the past call history of

On the other hand, Patent Document 2 discloses a distributed call recording system for making a large number of outbound calls made by an operator to a customer.
JP, 2006-94260, A JP, 2005-210227, A

  By the way, in general, in the call center, the call reception is widely targeted to customers who purchase almost all products manufactured and sold by business operators, and in some cases, not only to the product purchasers but also to general consumers. For this reason, the call center operator can overwhelm the problem of the product of the business operator excessively or criticize the customer's attitude or service failure, or the complaint call from the so-called "cremer" I can not but answer the phone.

  However, the calls from such harmful customers often occupy calls for a long time, and furthermore, many calls are made repeatedly and frequently, which hinders the operator's smooth telephone service and the operator's call. The efficiency of the response is significantly reduced. On the other hand, when responding to a threatening complaint from a harmful customer by telephone, if the response is wrong, this may be used as a paradox to be called for further unfair monetary compensation or to be extended to legal action. If business operators are exposed to litigation risk, and information disseminating the business operators' credit is disseminated on the Internet, their social trust may be lost. In particular, if the call center operator is not an employee but an outsourced contractor, it will be more difficult to simultaneously maintain the quality and efficiency of the operator's telephone response to harmful customers.

  For this reason, it is strongly required to notify and warn the call center operator at the initial stage of telephone service that the received call is a call from the above-mentioned harmful customer.

  Here, in the case of a normal customer call, when the operator receives a call, the operator is made to tell the customer one or more identifiers for identifying the customer name, address, product serial number, and other customers. By searching the customer database using the told identifier as a key, and displaying the retrieved customer information and past call history information on the operator terminal, who is the customer and what telephone call was answered in the past You can easily grasp what has been done. The customer name can also be identified by searching the customer database using the caller telephone number obtained from the communication carrier as a key.

  However, the above-mentioned harmful customer has a behavior and behavior pattern that tries to avoid being identified as a harmful customer who has filed a complaint in the past, as the content of the complaint is worse. For this reason, even if the operator asks for a name at the time of telephone reception, it is not possible to identify the harmful customer identified in the past by searching the customer database because the true name is not often given. In addition, harmful customers can use phones other than home phones and mobile phones registered in the customer database to avoid being identified as past harmful customers when making calls, or they can not use the caller's phone number. Calls are often made in notification mode. Therefore, the customer name can not be specified by referring to the customer database or by the caller telephone number obtained from the communication carrier.

  The present invention has been made in view of the above problems, and an object thereof is to provide a smooth telephone answering service in a CRM system for recording, storing, and managing calls made between a customer's telephone and a person in charge of the person in charge of the call. To provide a harmful customer detection system, method and a harmful customer detection program capable of detecting and warning such harmful customers in real time at the initial stage of telephone service.

  Another object of the present invention is to make it possible to identify that a speaker is a harmful customer in real time during a call, without being aware of the originating harmful customer himself.

  Another object of the present invention is that even a harmful customer who does not give a true customer name or a harmful customer who makes a call from a telephone other than a telephone registered at a call center, the speaker may be harmful in real time during the call. The point is to make it possible to identify the customer.

  Harmful customers in the context of call center operations generally have a history of having made complaint calls to the call center in the past when calling to the call center, but on the other hand, they do not name an authentic customer name, nor are registered phones It is characterized as difficult to identify by number.

  In the present specification and claims, the term “harmful customer” typically exaggerates and criticizes problems of the product of the business operator excessively, or excessively pursues the inconvenience of customer service and service. It is a customer, and so-called "cremers" are also included in this, but it is not limited to this, and broadly means any user who can take an action pattern that may hinder smooth telephone service operations. For example, so-called "suspicious persons" and the like can also be candidates for potential customers of harmful customers.

  As a technology that can be used to identify a speaker having such a feature, a person can utter a voiceprint by collating a voiceprint between the pre-registered speaker's recorded voice and the voice input at the time of authentication. A voiceprint matching technique is known which specifies whether a voice is voiced. However, since this voiceprint collation determines who among the N speakers who are registered in advance is the speaker to be identified by performing N comparison processes, computer processing is inevitable. In case the voiceprint of a considerable number (for example, 100 or more but not limited to) harmful customers is registered because of heavy load, is it likely that the harmful customers notice the matching of the input voice with the voiceprints of all cases? It is extremely difficult to realize in real time without being

  In the present invention, in order to execute the matching determination of the input voice with the registered voice in real time during a call at a call center, it is possible to withstand an increase in the number of registered persons in multiple stages (for example, two stages). Execute voice matching processing. Since it is sufficient for the person in charge of telephone calls to be alerted when the source speaker thinks that the customer is a harmful customer, the matching determination process may have practical accuracy. Moreover, the speech text of the input speech and the speech text of the speech registered in advance may not necessarily match.

  According to one aspect of the present invention, there is provided a harmful customer detection system for detecting in real time whether or not a speaker in a call corresponds to each of all pre-registered harmful customers, wherein the harmful customer detection system is specified in advance. A voiceprint information extraction unit which reads out the recorded speech signal of the speaker to be registered from the storage device and extracts voiceprint information characterizing the speaker to be registered, and calculates a relative distance of voiceprint information of all registered speakers. A voiceprint having a minimum total sum of relative distances between voiceprint information of other registered speakers belonging to each group and a grouping unit that groups the registered speakers having voiceprint information whose relative distance is within the first threshold. A reference person setting unit for setting a registered speaker having information as a reference person in the group, extracted voiceprint information, an identifier of the registered speaker, a group identifier to which the registered speaker belongs, a reference person Store with the information of A fingerprint database, a voice acquisition unit for acquiring a call voice of a caller, and extracting voiceprint information of the caller, voiceprint information of the caller with reference to the voiceprint database, A group selection unit which calculates a relative distance only with voiceprint information of a reference person of each group registered in the voiceprint database, and selects a group to which a reference person having voiceprint information having the smallest relative distance belongs; The relative distance between the voiceprint information of the source speaker and the voiceprint information of the registered speaker other than the reference person belonging to the selected group is calculated, and the calculation of the relative distance is performed for all subjects other than the reference person. Select a registered speaker having voiceprint information having the smallest relative distance in the selected group, and the relative distance being within a second threshold smaller than the first threshold, repeatedly for the registered speaker Speaker selection And a message output unit for outputting a warning message including the identifier of the selected speaker to be visible or voice-recognizable during a call by the called party talker with the caller. A hazardous customer detection system is provided.

  The harmful customer detection system further decreases the first threshold when the number of registered speakers grouped into one group by the grouping unit exceeds a predetermined number of registrants. And a group reconstruction unit for regrouping registered speakers having voiceprint information within a specified first threshold.

  The voiceprint database further stores together harmful customer rank information indicating the degree of harmful customers of the registered speaker in association with the extracted voiceprint information, and the message output unit selects the selected registered speech The first warning message may be output when the harmful customer rank information of the person is lower than a predetermined value, and the second warning message may be output when the harmful customer rank information is larger than the predetermined value.

  The above-mentioned harmful customer detection system further refers to a telephone number database storing the telephone installation history information of the user together with the telephone number of the telephone, and a caller telephone obtained from the caller call information with reference to the telephone number database. Performing a credit check of the caller based on the telephone laying history information of the number, and outputting the result of the credit check as a credit check message so that the called party can visually recognize or recognize the voice before the call starts And a second message output unit.

  The harmful customer detection system may further include an emotion analysis unit that detects occurrence of fluctuation of the input call voice and notifies a message output unit.

  According to another feature of the present invention, a voiceprint information extraction unit, a grouping unit, and a criterion for detecting in real time whether a speaker in a call corresponds to each of all harmful customers registered in advance. A harmful customer detection method executed by a harmful customer detection system including a person setting unit, a voiceprint database, a voice acquisition unit, a group selection unit, a registered speaker selection unit, and a message output unit, A voiceprint information extraction step of reading out from the storage device the recorded voice signal of the registered speaker identified in advance and extracting voiceprint information characterizing the respective registered speaker, and a relative distance between voiceprint information of all registered speakers The sum total of relative distances between the grouping step of calculating and grouping the speakers to be registered having voiceprint information whose relative distance is within the first threshold, and the voiceprint information of the other registered speakers belonging to each group minimum And a reference person setting step of setting a registered speaker having voiceprint information as a reference person in the group, an extracted voiceprint information, an identifier of the registered speaker, a group identifier to which the registered speaker belongs, A voiceprint information storing step of storing the voiceprint database together with the information of the reference person; a voice obtaining step of obtaining a call voice of the caller; and extracting voiceprint information of the caller; and referring to the voiceprint database. The relative distance is calculated only between the voiceprint information of the originating speaker and the voiceprint information of the reference person of each group registered in the voiceprint database, and a criterion having voiceprint information with the smallest relative distance. Between the group selection step of selecting the group to which the subject belongs, the voiceprint information of the originating speaker, and the voiceprint information of the registered speaker other than the reference subject belonging to the selected group A relative distance is calculated, and the calculation of the relative distance is repeated for all registered speakers other than the reference person, and voiceprint information having the smallest relative distance in the selected group is included, and the relative distance is the first one. Selecting a speaker-to-be-selected for selecting a speaker-to-be-registered within a second threshold smaller than the threshold and a warning message including an identifier of the speaker-to-be-selected selected with the source speaker A method for detecting a harmful customer is provided, which includes a message output step of outputting a visual recognition or voice recognition during a call by a pre-speaker.

According to another feature of the present invention, the harmful for causing the computer to execute a harmful customer detection process which detects in real time whether or not a speaker in a call corresponds to each of all the harmful customers registered in advance. A customer detection program, wherein the program reads out from a storage device a recorded voice signal of a registered speaker specified in advance in the computer, and extracts voiceprint information extracting the voiceprint information characterizing the respective registered speaker And grouping processing for calculating the relative distance of voiceprint information between all registered speakers and grouping the registered speakers having voiceprint information whose relative distance is within the first threshold, and belonging to each group A reference person setting process for setting a registered speaker having voiceprint information having a minimum total sum of relative distances with voiceprint information of the registered speaker as a reference person in the group, and extracted voiceprint information The voiceprint information storage process stored in the voiceprint database together with the identifier of the registered speaker, the group identifier to which the registered speaker belongs, and the information of the reference person, and the call voice of the calling speaker is acquired, Acquisition processing for extracting the voiceprint information of the person, and referring to the voiceprint database, only between the voiceprint information of the originating speaker and the voiceprint information of the reference person of each group registered in the voiceprint database Group selection processing for calculating a relative distance and selecting a group to which a reference person having voiceprint information having the smallest relative distance belongs, voiceprint information of the source speaker, and a selected person belonging to the selected group Calculating the relative distance to the voiceprint information of the registered speaker, and repeating the calculation of the relative distance for all registered speakers other than the reference person, and the relative distance is the smallest within the selected group Registered speaker selection processing for selecting a registered speaker having voiceprint information and whose relative distance is within a second threshold smaller than the first threshold, and an identifier of the selected registered speaker And a message output process for outputting a warning message including the message by the called party who talks with the calling party during the call so as to allow the computer to execute a process including a message output process. A characterizing harmful customer detection program is provided.

  According to the present invention, the voiceprint analysis server executes only matching between the reference speaker of each group and the input voice accepted by telephone from among the plurality of harmful customers' voice models registered and grouped in advance. The input speech is compared with the speech model belonging to the group to which the reference speaker determined as the most similar speech belongs, and the speech model having the similarity to the input speech within a predetermined threshold value is identified, and this speech is identified. Identify the offending customers that correspond to the model and notify the correspondent who answers the call during the call.

  As a result, in the CRM system that records, manages, and manages calls made between the customer's phone and the person in charge of the handling staff, harmful customers who impede smooth phone handling operations can be real-timed at the initial stage of phone handling. Can be detected and warned.

  Also, it is possible to identify that the utterer is a harmful customer in real time during a call, without being aware of the harmful customer of the originator.

  Furthermore, even in the case of a harmful customer who does not give a true customer name or a harmful customer who makes a call from a phone other than a telephone registered at the call center, the speaker identifies the harmful customer in real time during the call. can do.

  Therefore, according to the harmful customer detection system, the method thereof and the harmful customer detection program according to the present invention, in the call center operation, it is possible to automatically grasp harmful customers at the initial stage of telephone service without requiring additional facilities. This will contribute to the improvement of the CRM of business operators, because it will improve the efficiency of telephone handling operations.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functions and configurations will be denoted by the same reference numerals and redundant description will be omitted.

<Configuration of this embodiment>
FIG. 1 shows one non-limiting example of a network configuration of a harmful customer detection system according to an embodiment of the present invention. The harmful customer detection system includes a PBX (switchboard) 1, a voice acquisition server 2, a voice print analysis server 3, a telephone number analysis server 4, an emotion analysis server 5, a control server 6, a customer telephone terminal 7, a PSTN (public telephone network) 8, The operator telephone terminal 9a, the operator PC terminal 9b, and the call recording server 10 are provided. Among the harmful customer detection system, PBX (switchboard) 1, voice acquisition server 2, voice print analysis server 3, telephone number analysis server 4, emotion analysis server 5, control server 6, operator telephone terminal 9a, operator PC terminal 9b, call recording server All or a part of 10 may be installed in a call center and interconnected by an IP (Internet Protocol) network such as an intranet 11 d such as a LAN / WAN. Alternatively, alternatively, the voice acquisition server 2, the voice print analysis server 3, the telephone number analysis server 4, the emotion analysis server 5, the control server 6, the call recording server 10, and the call recording database 101, the voice print database 31, these telephones All or part of the number database 41 or the like may be installed outside the call center as appropriate via a remote IP connection such as the Internet.

  The PBX 1 connects the extension telephones in the call center and connects each operator telephone terminal 9a to the PSTN (public telephone network) 8 via the private lines 11a, 11b, 11c,... A call between the telephone terminal 9a and the customer telephone terminal 7 is realized.

  The voice acquisition server 2 is branch-connected to the PBX 1 and acquires call voices between each operator telephone terminal 9a and the customer telephone terminal 7, and associates the acquired voice with the number (for example, extension number) of the operator telephone terminal 9a. Supply to each server.

  Alternatively, this voice acquisition server 2 may be branch-connected to the circuit between the termination unit (DSU) of the PSTN 8 and the PBX 1.

  The voiceprint analysis server 3 compares the acquired voice supplied from the voice acquisition server 2 with the voice model (voiceprint pattern) of the harmful customer registered in advance in the voiceprint database 31, and the comparison result is an extension of the operator's phone terminal 9a. It supplies the control server 6 with the number.

  The telephone number analysis server 4 uses the telephone number of the customer telephone terminal 7 of the transmission source supplied from the PBX 1 as a key, and in advance performs a credit check obtained by searching the telephone number database 41 for registering credit information for each customer telephone number. The result is supplied to the control server 6 together with the extension number of the operator's telephone terminal 9a.

  The emotion analysis server 5 controls the emotion analysis result obtained by detecting fluctuation in voice of the acquired voice supplied from the voice acquisition server 2, the height of emotion, restlessness, etc. along with the extension number of the operator's telephone terminal 9a. Supply to 6.

  The control server 6 is based on the comparison result supplied from the voice print analysis server 3, the credit check result supplied from the telephone number analysis server 4, and all or part of the emotion analysis result supplied from the emotion analysis server 5. It is determined whether the acquired voice is a harmful customer or not, and the determination result is displayed as a warning on the operator PC terminal 9b in real time. Preferably, the analysis results may be alerted along with rank information indicating the extent of harmful customers. Preferably, the operator PC terminal 9b has a browser function, and the recorded voice data stored in the call recording database 101, and user information or harmful customer information stored in the telephone number database 41 to the voiceprint database 31. Can be searched and displayed as appropriate. Alternatively, or in addition to this, the control server 6 tones the operator's telephone terminal 9a with a warning sound not heard by the customer who is the caller when the caller during the call corresponds to a harmful customer, etc. Voice output may be interrupted.

  Under control of the control server 6, the call recording server 10 is a call recording that includes acquired voice supplied from the voice acquisition server 2 after an incoming call, for example, by a large scale external storage device such as NAS (Network Appliance Storage). It is accumulated and saved in the database 101.

  Although the PBX 1 in FIG. 1 is connected to the customer call terminal 4 via a public switched telephone network such as the PSTN 1 or the like, instead of this or in addition to this, by providing an IP network connection function, The voice acquisition server 2 may be connected to a customer IP call terminal having an IP telephone function via a voice packet communication network such as a Voice Over Internet Protocol (VoIP) network, and in this case, the voice acquisition server 2 comprises a customer IP call terminal and an operator phone terminal. A voice call between 9a can be obtained. The customer telephone terminal 7 may be either a fixed telephone or a mobile telephone.

  The voiceprint information extraction unit, grouping unit, and reference person setting unit in the claims refer to the voiceprint analysis server 3 to the control server 6, and the voice acquisition unit in the claims refers to the voice acquisition server 2 to the control server 6, in the claims. The group selection unit and the registered speaker selection unit correspond to the voiceprint analysis server 3, and the message output unit in the claims corresponds to the control server 6 or the operator terminal device 9b.

  Further, the configuration of the network and hardware shown in FIG. 1 is merely an example, and each server and database may be integrated as necessary, or each component may be installed outside an application service provider (ASP) or the like. .

<Overview of Hazardous Customer Detection Processing in this Embodiment>
FIG. 2 shows a customer who is in a call from an incoming call from the customer telephone terminal 7 to the operator telephone terminal 9a of the call center in the harmful customer detection system according to the first embodiment, which is executed under the control of the control server 6. A non-limiting example of a processing sequence for notifying the operator terminal 9b to the telephone terminal 9a that it is a harmful customer and performing distribution processing is shown.

  In FIG. 2, the customer telephone terminal 7 first receives a call from the customer telephone terminal 7 to the PBX 1 (step S1). The PBX 1 obtains call information by an incoming call. The acquired call information includes, for example, incoming call start information (including incoming call start time stamp), outgoing call start information (including outgoing call start time stamp), call start information (including call start time stamp), call end information ( Call control information such as call end time stamp), source phone number, destination phone number, source channel number, sender number, incoming channel number, incoming phone number (destination extension number etc) And identification information, preferably in conjunction with a CTI program operating on the control server 6 implementing the Computer Telephony Integration (CTI) protocol or on the operator PC terminal device 9b, on these display devices. May be displayed in real time. The PBX 1 associates all or part of the acquired call information, at least the number of the customer telephone terminal 7 of the transmission source, and the extension number of the operator telephone terminal 9a which has been off hook, and this transmission source telephone number and the operator extension number And the telephone number analysis server 4 (step S2). Preferably, the telephone number analysis server 4 further obtains, from the call information, a flag indicating whether or not the calling customer's telephone terminal has not notified the calling party number.

  FIG. 3 shows an example of the configuration of the telephone number database 41 referred to by the telephone number analysis server 4. In FIG. 3, the telephone number database 41 includes a user database 411 and a telephone installation database 412. The user database 411 is configured, for example, in a table format, and as an example, for each user, a user ID for uniquely identifying a user (customer), a user name, a telephone number in the telephone installation database 412 Link to, telephone number change history information, telephone charge delinquent history information. The telephone installation database 412 is configured, for example, in a table format, and includes, for example, information of a telephone number and a telephone installation date for each telephone number. Alternatively, the user database 411 and the telephone installation database 412 may be integrally configured using, for example, a telephone number as a key.

  Returning to FIG. 2, the telephone number analysis server 4 searches the telephone number server 41 using the acquired source telephone number as a key to perform telephone number analysis processing based on the source telephone number, that is, a simple credit check processing. (Step S3), and the credit check result is output to the operator PC terminal 9b corresponding to the acquired operator extension number via the control server 6 (step S4).

  As an example, the telephone number analysis server 4 refers to a flag indicating whether or not the caller telephone number has been notified, and refers to the telephone installation date to be notified of the caller telephone number, and the caller telephone If the predetermined period of time has been laid (that is, if the contract period of the fixed telephone or the mobile telephone has been long), it is determined that there is no abnormality as the credit check result. On the other hand, if the caller ID is notified but the originator's phone is installed soon (ie, the contract term of the fixed telephone or the mobile phone is short), or the caller ID is not notified. If it is, it is judged that there is a warning as a credit check result.

  Preferably, when it is determined that the warning is present, the telephone number analysis server 4 extracts the voiceprint information of the recorded voice as the potential person of the suspicious person and registers the voiceprint information in the voiceprint database 31. The control server 6 may be instructed to do this. In addition, preferably, a plurality of warning levels are set in advance, for example, when the caller number is not notified, the caller number is notified but the caller telephone has been installed for a while. You may raise the warning level more than the case. Before the call between the customer and the operator is started, the operator PC terminal device 9b can display a warning about the possibility of a harmful customer, which can alert the operator before the call starts ( Step S5).

  Next, a voice call is started between the customer phone terminal 7 and the operator phone terminal 9a (steps S6 and S7), and the voice is acquired by the voice acquisition server 2 and supplied to the voiceprint analysis server 3.

  FIG. 4 shows an example of the configuration of the voiceprint database 31 referred to by the voiceprint analysis server 3. In FIG. 4, the voiceprint database 31 includes a harmful customer database 311 and voiceprint group data 312. The harmful customer database 311 is configured, for example, in tabular form, and for example, for each customer identified beforehand as a harmful customer, a harmful customer ID for uniquely identifying the harmful customer, harmful customer personal information, harmful customer rank, A link to voiceprint group data and information of a group ID of voiceprint information to which the harmful customer belongs are included. The voiceprint group data 312 is formed by grouping voiceprint data having a high degree of similarity in advance, and is configured for each voiceprint group, includes voiceprint data of harmful customers belonging to the voiceprint group, and each voiceprint data is an individual A link to the information is associated with information that accounts for whether the harmful customer having the voiceprint data is a reference person in the group.

  Returning to FIG. 2, the voiceprint analysis server 3 cuts out a voice acquired by telephone call for a predetermined time (for example, 3 seconds or more), and voiceprint data (voiceprint pattern to be described later) of this clipped acquired voice The voiceprint data of the harmful customer registered in advance is compared with it to determine whether the speaker of the acquired voice corresponds to the registered harmful customer (step S8), and this voiceprint analysis result is controlled It outputs to the operator PC terminal 9b corresponding to the acquired operator extension number via the server (step S9). Preferably, the output voiceprint analysis result includes the detected harmful customer name, the harmful customer rank indicating the degree of harmfulness, and when the operator selects the displayed harmful customer name etc., the past of the harmful customer The call history and the telephone correspondence history may be displayed on the operator PC terminal 9b continuously. Alternatively, the input voice and a plurality of harmful customer names whose similarity is within a certain threshold may be displayed as the corresponding harmful customer candidate list. Details of this voiceprint analysis process will be described later.

  Next, the emotion analysis server 5 detects, through the control server 6, an emotion analysis result obtained by detecting fluctuation in voice of acquired voice supplied from the voice acquisition server 2, height of emotion, restlessness and the like. Then, the information is output to the operator PC terminal 9b corresponding to the acquired operator extension number (steps S10 and S11). By displaying the emotion analysis result of the input voice on the operator PC terminal 9b, it is possible to identify as a potential harmful customer at the initial stage of telephone correspondence even a customer who has no history of claiming a complaint in the past.

  The telephone number analysis result from the telephone number analysis server 4, the voice print analysis result from the voice print analysis server 3, and the emotion analysis result from the emotion analysis server 5 are transmitted to the operator PC via the control server 6 each time a result is obtained. Alternatively, or in addition to this, the control server 6 synthesizes the integrated judgment result on the basis of the plurality of analysis results, for example, the speaker currently in a call is a harmful customer If it is determined that there is a problem, it may be displayed on the operator PC terminal 9b as a pop-up message or the like along with a message notifying that the customer in a call corresponds to a harmful customer. Preferably, the customer information stored in the telephone number database 41 and the voiceprint database 31 is referred to on the operator PC terminal 9b together with the alerting message or in response to the operator's display request in response to the message output. For example, a customer name, an address, a registered telephone number, a current caller telephone number, a result of a voiceprint check, a customer attribute, a product purchase history, and the like may be displayed.

  Preferably, for example, when a similar registered harmful customer name is not obtained, a normal telephone correspondence is instructed or no message is displayed, and a similar registered harmful customer name is obtained. In accordance with the obtained harmful customer rank, if the harmful customer rank is low, the operator in the call is instructed to handle the telephone call for harmful customers according to the manual, and if the harmful customer rank is high, a particularly careful response is required. The call may be transferred to the supervisor of the operator who is in a call, a skilled operator, or the like, or alternatively, a warning message may be directly displayed on the PC terminal or the like of these supervisors to prompt an interrupt call (step S12).

  As a modification, when a call is made to the operator's telephone terminal 9a, for example, the operator responds to the telephone after outputting an automatic response voice to the source telephone terminal using a known IVR (Interactive Voice Response) function. May be configured to start. For example, when receiving an incoming call, the automatic response voice prompts the originating customer to utter the contents of the inquiry and the customer name etc. In response to this, the voice print analysis result and emotion analysis result of the input voice are obtained, and the operator PC After being displayed on the terminal device 9b, if the operator starts telephone answering, it is possible to execute the voiceprint check in more detail without causing the originating customer to be unnatural, for example, the customer name and inquiry By performing a voiceprint check between a predetermined utterance content such as the content and a registered voiceprint pattern (voice model) in which the same utterance content is templated, it is possible to improve the accuracy of voiceprint analysis.

  As another modification, the voiceprint analysis server 3 transmits the voiceprint source only when it is determined by the phone number analysis server 4 that “the alert is present” for the source speaker as a result of the telephone number analysis process based on the source phone number. This embodiment may be configured such that the voiceprint analysis process of the speaker is executed, and the voiceprint analysis process by the voiceprint analysis server 3 is not performed when it is determined that "the abnormality is not present" for the source speaker. According to this configuration, the load on the system related to voice print analysis at the time of telephone response with a customer who is determined to be "no abnormality" can be alleviated, and the operator's telephone response can be speeded up.

<Injury customer voiceprint data registration process in the present embodiment>
FIG. 5 shows an example of a processing procedure for registering the voice model of the harmful customer according to the first embodiment in the voiceprint database 31 in advance.

  In FIG. 5, the call voice data of a specific speaker to be registered as a harmful customer is read out from the call voice data recorded in the past from the call recording database 11 (step S31), and the call of this specific speaker is made. Voiceprint information (data of voiceprint pattern) indicating the feature is extracted from the voice data (step S32). The identification of the speaker to be registered as the harmful customer may be specified by the operator who made the telephone call, or may be automatically selected based on a keyword in the call history, personal information, and the like. Alternatively, for example, personal information may be newly recorded after identification of the registered speaker is confirmed in a telephone response between the operator and the registered speaker.

  In the present specification, “personal information” is personal information of a customer, and includes, for example, a customer name, an address, a registered telephone number, a date of birth, a customer attribute, a product purchase history, etc. It can be appropriately displayed on the terminal 9b together with a warning or in response to an instruction input by the operator. Further, as the customer attribute, for example, “hazardous customer (plus harmful customer rank)”, “important customer (VIP)” and the like can be identified. Furthermore, although a speech model necessary for matching with the input speech can be generated from speech uttered for, for example, about 10 seconds, the required number of seconds is the processing capacity of the processor, the capacity and data of the external storage device, etc. It may vary depending on the structure.

  Here, a speaker recognition technology that can be used by the voice model analysis process and the voice print analysis process executed by the voice print analysis server 3 and the control server 6 in this embodiment will be described, in particular, a speaker identification technology. Unlike speech recognition processing that recognizes what a speaker is speaking, what is identified by the speaker's voice is speaker recognition, and in this speaker recognition, it is speaker recognition. And acoustic patterns extracted from human speech, gender characteristics of men and women, anatomical features such as throat and mouth size and shape, speech speed and style, etc. extracted as voice print information and modeled Recognize the voice of the individual. In this speaker recognition, in the present embodiment, the voice of the input speaker is compared with each of a large number of stored voice models to identify the voice whose voice does not know who the voice is. Speaker identification (Speaker Identification). Further, in the speaker identification according to the present embodiment, since it is necessary to record the speech and extract the voiceprint information for comparison without the harmful customer being aware of it, the speech content of the speaker is limited in advance. It is difficult. Therefore, the speaker identification is performed based on the general voice feature of the speaker, assuming that the speech text of the recorded speech from which the voiceprint information is extracted matches the speech text of the input speech as well as the case where they do not coincide. .

  The details of the process of extracting voiceprint information from the recorded voice will be described with reference to FIGS. 6 to 12. The recorded speech is labeled in phoneme units. That is, if it is a vowel in the phoneme unit, it is "a", "i", "u", "e", "o", and if it is a consonant, "k", "s", "t", " n "," h "," m "," j "," r "," w "," g "," z "," d "," b "," p "are given as labels Ru. FIG. 6 is a graph showing the magnitude of air vibration on the vertical axis and the time on the horizontal axis in the case of sounding “denshi”, and a different waveform pattern appears for each phoneme.

  Next, the labeled speech is normalized (unified) according to the pitch, size, and speed of the sound. Since the voice differs in pitch (voice pitch) depending on the individual, as shown in FIG. 7, the pitch is changed by thinning the original signal and further reducing the time axis to normalize the pitch. Do. Further, since the voice differs in size and speed for each utterance, as shown in FIG. 8, the voice waveform is expanded or contracted to normalize the voice size and speed so as to match the speed of the reference voice. These normalization processes are performed on a phoneme basis.

  Next, voiceprint information is extracted by generating a sound spectrogram. The spectrum of speech frequencies characterizes the speaker's voiceprint. This frequency spectrum can be obtained by Fourier transforming a time signal, for example, applying Fast Fourier Transform (FFT) processing suitable for processing by a processor to a time waveform signal of speech. Can be realized by FIG. 9 shows the frequency (Hz) on the horizontal axis and the magnitude (dB) on the vertical axis of how much frequency a certain sound contains by applying FFT processing to a speech signal. The speech spectrum shown in the graph, that is, the speech spectrum is shown.

  FIG. 10 shows an envelope on the speech spectrogram, in which a plurality of peaks appear. Each of the peaks on the speech spectrum is called a formant, and the change on the time axis of the formant indicates the voice print characteristic derived from the individual difference.

  When a human utters a voice continuously, individual differences appear in the voice. For example, the speech of a good speaker of syntactic tongues has a clear separation of tones, and the speech of a poor speaker of speech tongues has an unclear separation of tones. If the change on the time axis of the speech spectrum is observed, FIG. 11 shows the speech signal uttered as "denshi" with time on the X axis, frequency on the Y axis, and magnitude of sound on the Z axis. The graph of the spectrum at each point in time, while shifting the time gradually, shows the three-dimensional change of the speech spectrum on the time axis, and FIG. 12 shows the magnitude of the time on the Z axis. The three-dimensional graph of FIG. 11 is converted to a two-dimensional display by indicating the thickness by the shade of color. Referring to FIG. 12, it is understood that patterns are different for each phoneme. FIG. 13 shows the voice spectrogram obtained in FIG. 12, that is, the voiceprint pattern, showing the individual difference between the pattern of the speaker A and the pattern of the speaker B and showing the individual differences in this embodiment. The voiceprint collation process executed by the analysis server 3 calculates which of the voiceprint patterns of the registrant registered in advance the voiceprint pattern of the input voice is most similar.

  Returning to FIG. 5, the thus obtained spectrum of each phoneme is registered in the voiceprint database 31 as voiceprint information together with the personal information of the registrant (step S33).

<Noxious customer voice model grouping processing and update processing in the present embodiment>
In FIG. 5, for each speaker (registered harmful customer) registered in the voiceprint database 31, a similarity index with other registered harmful customer is calculated (step S34).

  FIG. 14 shows a schematic example of the similarity index calculated based on the spectrum signal on the time axis for each phoneme of the registrant registered in the voiceprint database 31. For example, the registrants A to K in the voiceprint database Assuming that is registered, the similarity index calculated by scaling in the range of 0 points to 100 points is described on the matrix between each registrant. FIG. 15 shows an example of calculating the relative distance of voiceprint patterns between registrants by calculating (100-similarity index) based on the similarity index calculated in FIG. FIGS. 16 and 17 are based on the relative distance of voiceprint patterns between registrants obtained in FIG. Show the results of For example, the relative distances between the registered speakers A, B, C, D, and E are −40 ≦ AB ≦ 40, −40 ≦ AC ≦ 40,..., −40 ≦ DE Since ≦ 40, the relative distances between the registered speakers A, B, C, D, and E are within 40 points which is the grouping threshold, and this is regarded as group 1. Similarly, the relative distances between the registered speakers F, G, H, I, and J are −40 ≦ F−G ≦ 40, −40 ≦ F−H ≦ 40,..., −40 ≦ I− Since J ≦ 40, the relative distance absolute values of the registered speakers F, G, H, I, J are within 40 points which is the grouping threshold, and this is regarded as the group 2. With respect to the registered speaker K, since all relative distances to other registered speakers exceed the grouping threshold (55 to 75), K alone is set to the group 3.

  Returning to FIG. 5, the absolute value of the relative distance of the voiceprint pattern (voiceprint information) between the other registered speakers (in the present specification, unless otherwise specified, the “relative distance” indicates the absolute value of the relative distance ) Groups the registrants who are within the grouping threshold (step S35).

  Next, a registered speaker with the smallest total sum of relative distances with other in-group registered speakers in each group is set as a reference person of the group (step S36). For example, in the example shown in FIGS. 14 to 17, the in-group 1 reference person is B, the in-group 2 reference person is H, and the in-group 3 reference person is K. The information of the group thus obtained and the information of the reference person of each group are registered in the voiceprint database 31 (step S37).

  Here, when the number of registrants belonging to one group exceeds a predetermined number of registered persons in a group threshold, for example, 10 as an example, the number of registrants belonging to one group (the number of speakers in the group The threshold value for grouping of the relative distance (absolute value) of the voiceprint pattern is decreased so that the threshold value is within 10 persons (step S38). As described above, by providing a threshold value for the number of registrants belonging to one group and maintaining the number of registrants in the group within a predetermined number, voiceprint collation performed for all registrants in one group It is possible to keep the required time within a predetermined time, and to maintain the real-time property in the input voiceprint collation.

  In the above example, the grouping threshold is 40 points and the in-group speaker number threshold is 10, but in order to determine these thresholds, the total time required for speaker identification according to the present embodiment is determined. It is preferable to make a decision based on whether or not the response interval time allowed during a call is exceeded. As an example, for example, the response interval time allowed during a call is within 5 seconds, more preferably within 2 seconds, and comparison processing between voiceprint information of one registered speaker and input voiceprint information If it takes 0.2 seconds, it is acceptable within a practical time to perform comparison processing for a total of 10 people, so (number of groups + (number of inter-group speakers threshold -1)) is 10 The threshold value for the number of inter-group speakers may be determined so that the relative distance grouping threshold value may be determined based on this.

<Voice print collation processing in the present embodiment>
FIG. 18 shows an example of the processing procedure of voiceprint collation of input voice according to the present embodiment.

  In FIG. 18, voiceprint information of a reference person of each group (registered users B, H, and K in the example of FIGS. 16 and 17) and voiceprints of input speech of a source speaker whose operator is answering a telephone call The collation with the information is executed (step S61). In order to perform this voiceprint collation, the input voice is preferably recorded, for example, for about 3 seconds, and the voice signal is labeled for each phoneme, as already described in the registration process to the voiceprint database 31. Speech spectograms (voiceprint patterns) are extracted after being normalized with respect to pitch, size, and speed.

  As a result of the voiceprint collation process with the reference person in step S61, the group to which the reference person who has the closest relative distance to the voiceprint information of the input voice (that is, the largest similarity index) belongs is selected (step S62).

  In the selected group, the voiceprint information of all registered speakers except the reference person who has already been subjected to voiceprint verification is compared with the voiceprint information of the input voice (step S63).

  As a result of the voiceprint collation, the relative distance of the input speech to the voiceprint information is the smallest, with all registrants in the selected group (reference person + registrants targeted for collation in step S63) as a population And, a registrant having voiceprint information within the coincidence determination threshold is specified (step S64). The match determination threshold is preferably a value smaller than the grouping threshold (for example, 40 points) described above, and may be, for example, 20 points.

  Thus, the voiceprint analysis server 3 sends the harmful customer identifier such as the harmful customer name identified in step S64 to the control server 6 together with the registered personal information. On the other hand, when the registrant is not specified in step S64, the voiceprint analysis server 3 sends data indicating that there is no corresponding person to the control server 6. The control server 6 displays, for example, the received harmful customer name and the harmful customer rank on the operator PC terminal 9b, and preferably, in response to an instruction input from the operator, the user further uses the received harmful customer identifier as a key The database and the call recording database are referred to as appropriate, and the past correspondence history, call history and the like are displayed on the operator PC terminal 9b.

<Hardware configuration of the harmful customer detection system according to the present embodiment>
FIG. 19 is a block diagram showing an example of the hardware configuration of each server device according to the present embodiment. In each server device, which is the computer device 110 shown in FIG. 19, the CPU 111 controls the entire system according to a program stored in the ROM 114 and / or the hard disk drive 116 using the RAM 115 as a primary storage work memory. Furthermore, the CPU 111 executes the harmful customer detection process according to the first embodiment based on a program stored in the hard disk drive 116 in accordance with a user instruction input via the mouse 112 a or the keyboard 112. A display such as a CRT or LCD is connected to the display interface 113, and an input standby screen for harmful customer detection processing executed by the CPU 111, processing progress, processing results, search results and the like are displayed. The removable media drive 117 is mainly used when writing a file from the removable media to the hard disk drive 116 or writing a file read from the hard disk drive 116 to the removable media. As removable media, floppy disk (FD), CD-ROM, CD-R, CD-R / W, DVD-ROM, DVD-R, DVD-R / W, DVD-RAM or MO, or memory card, CF Cards, smart media, SD cards, memory sticks, etc. can be used.

  A printer such as a laser beam printer or an ink jet printer is connected to the printer interface 118. The network interface 119 is an interface for connecting a computer device to a network.

  The input means for each server device and the operator PC terminal 9b according to the first embodiment is not limited to the mouse 112a or the keyboard 112, and any pointing device such as a track ball, a track pad, a tablet, etc. It can be used. When the portable information terminal is used as an input / output device connected to the server device according to the present embodiment and the operator PC terminal 9b, the input unit may be configured by a button, a mode dial, or the like.

  Further, the hardware configuration of each server according to the present embodiment shown in FIG. 19 is merely an example, and it is needless to say that any other hardware configuration can be used.

  In particular, all or part of the harmful customer detection process according to the present embodiment may be realized by the computer terminal device 110 or a portable information terminal device such as a PDA, etc. The Internet or any known local area interconnected by a wireless communication such as (registered trademark) or a wired communication line such as the Internet (TCP / IP), public telephone network (PSTN), integrated services digital network (ISDN), etc. Some or all of the call recording process may be implemented by a network system consisting of a network (LAN) or a wide area network (WAN).

  As described above, according to the present embodiment, in the CRM system for recording, storing and managing the calls made between the customer's telephone and the telephone of the person in charge, the harmful customer who impedes the smooth telephone correspondence operation is It can detect and alert in real time at the initial stage of telephone service.

  Also, it is possible to identify that the utterer is a harmful customer in real time during a call, without being aware of the harmful customer of the originator.

  Furthermore, even in the case of a harmful customer who does not give a true customer name or a harmful customer who makes a call from a phone other than a telephone registered at the call center, the speaker identifies the harmful customer in real time during the call. can do.

  The scope of the present invention is not limited to the illustrated and described exemplary embodiments, but also includes all the embodiments that bring about the same effects as the object of the present invention, without departing from the gist thereof. Various improvements or changes are possible within the scope. For example, the telephone number analysis process, the voice print analysis process, and the emotion analysis process disclosed in the present embodiment may be implemented alone in the harmful customer detection system according to the present embodiment, or implemented in any combination. It is also good.

  The present invention can also be applied to the operator identifying the key customer (VIP) at the initial stage of the telephone service. In this case, as "customer attribute", an attribute value indicating "important customer" and its content is provided, and as a result of the identification of the source speaker, for example, operator PC terminal 9b or the like indicates the importance customer. Just display the message. For example, assuming that the originating speaker simply names "Suzu-chi," the result of the speaker identification enables the operator to immediately answer telephone calls such as "You are Mr. Ichiro Suzuki of ○○ Shoji." Furthermore, the scope of the present invention is not limited to the combination of the features of the invention as defined by claim 1, but can be defined by any desired combination of particular features of all the disclosed respective features. .

It is a block diagram showing an example of the network configuration of the harmful customer detection system concerning one embodiment of the present invention. It is a figure showing control of a harmful customer detection processing, data flow, and time sequence which are performed by a harmful customer detection system concerning one embodiment of the present invention. It is a schematic diagram which shows an example of the data structure in the telephone number database 41 in FIG. It is a schematic diagram which shows an example of the data structure in the voiceprint database 31 in FIG. It is a flowchart which shows an example of the process sequence of the harmful customer voiceprint information registration process which the voiceprint analysis server 3 and control server 6 which concern on one Embodiment of this invention perform. It is a schematic diagram explaining the labeling process in a phoneme unit with respect to a recording speech. It is a schematic diagram explaining the pitch unification process with respect to a recording audio | voice. It is a schematic diagram explaining the magnitude | size and speed unification process of an audio | voice with respect to a recording audio | voice. It is a schematic diagram explaining the process which applies a fast Fourier transform to the time signal of sound recording audio | voice, and obtains an audio | voice spectrogram. It is a schematic diagram explaining the process which detects a formant from the envelope with respect to audio | voice spectrogram. It is a schematic diagram explaining the change on the time-axis of an audio | voice spectrogram. It is the figure which converted and displayed the audio | voice spectrogram three-dimensional graph of FIG. 11 into a two-dimensional graph. It is a figure explaining the difference by the individual difference of the audio | voice spectrogram of FIG. 12 as an example. It is a table | surface which demonstrates the similarity index calculated about the speakers registered into the voiceprint database as an example. FIG. 15 is a table for explaining as an example the relative distance between speakers calculated based on the similarity index calculated in FIG. 14. It is a graph which shows typically the relative distance between the speakers in FIG. FIG. 17 is a table for explaining an example of grouping results of speaker voiceprint patterns whose relative distance is within a predetermined grouping threshold, based on the graph of FIG. 16. It is a flowchart which shows an example of the process sequence of the voiceprint collation process which the voiceprint analysis server 3 which concerns on one Embodiment of this invention performs. It is a figure which shows an example of the hardware constitutions of each server apparatus which concerns on this embodiment.

Explanation of sign

PBX 1
Voice acquisition server 2
Voice print analysis server 3
Telephone number analysis server 4
Emotion analysis server 5
Control server 6
Customer phone 7
PSTN 8
Operator telephone terminal 9a
Operator PC terminal 9b
Call recording server 10
Public lines 11a, 11b, 11c
Voiceprint Database 31
Telephone number database 41
Call recording database 10

Claims (12)

A harmful customer detection system that detects in real time whether a speaker in a call corresponds to each of all registered harmful customers in advance,
A voiceprint information extraction unit which reads out from the storage device a recorded voice signal of a registered speaker identified in advance and extracts voiceprint information characterizing each of the registered speakers;
A grouping unit that calculates relative distances of voiceprint information between all registered speakers, and groups the registered speakers having voiceprint information whose relative distances between the registered speakers are all within the first threshold. When,
A reference person setting unit for setting, as a reference person in the group, a registered speaker having voiceprint information in which a total sum of relative distances with voiceprint information of other registered speakers belonging to each group is minimum;
A voiceprint database storing the extracted voiceprint information together with the identifier of the registered speaker, the group identifier to which the registered speaker belongs, and the information of the reference person;
A voice acquisition unit that acquires a call voice of a caller, and extracts voiceprint information of the caller;
The relative distance is calculated only between the voiceprint information of the originating speaker and the voiceprint information of the reference person of each group registered in the voiceprint database with reference to the voiceprint database, and the relative distance is the largest. A group selection unit for selecting a group to which a reference person having small voiceprint information belongs;
The relative distance between the voiceprint information of the source speaker and the voiceprint information of the registered speaker other than the reference person belonging to the selected group is calculated, and the calculation of the relative distance is performed for all subjects other than the reference person. Select a registered speaker having voiceprint information having the smallest relative distance in the selected group, and the relative distance being within a second threshold smaller than the first threshold, repeatedly for the registered speaker A speaker selection unit to be registered,
A message output unit for outputting a warning message including the identifier of the selected speaker to be displayed visually or voice-recognizable during a call by the called party who talks with the source speaker;
When the number of registered speakers grouped into one group by the grouping unit exceeds a predetermined number of registered users, the number of registered speakers in the group selected within the allowable call response interval time And a group reconstruction unit for decreasing the first threshold to complete selection processing, and regrouping a registered speaker having voiceprint information within the reduced first threshold. Harmful customer detection system. The voiceprint database further stores together harmful customer rank information indicating the degree of harmful customers of the registered speaker in association with the extracted voiceprint information.
The message output unit generates a first warning message when harmful customer rank information of the selected registered speaker is equal to or less than a predetermined value, and a second warning message when the harmful customer rank information is larger than the predetermined value. The harmful customer detection system according to claim 1, wherein: The above-mentioned harmful customer detection system further
A telephone number database for storing telephone installation history information of the user along with the telephone number of the telephone;
The credit check of the caller is executed based on the telephone installation history information of the caller telephone number obtained from the caller call information with reference to the telephone number database, and the result of the credit check is used as a credit check message. The harmful customer detection system according to claim 1 or 2, further comprising: a second message output unit that outputs a visual recognition or voice recognition by the destination speaker before the call starts. The above-mentioned harmful customer detection system further
The harmful customer detection system according to any one of claims 1 to 3, further comprising: an emotion analysis unit that detects occurrence of fluctuation of the input call voice and notifies a message output unit of the detection. A voiceprint information extraction unit, a grouping unit, a reference person setting unit, and a voiceprint information storage unit, which detects in real time whether or not a speaker in a call corresponds to each of all harmful customers registered in advance. A harmful customer detection method executed by a harmful customer detection system comprising a voice acquisition unit, a group selection unit, a registered speaker selection unit, a message output unit, and a group reconstruction unit,
Reading the recorded voice signal of the registered speaker identified in advance from the storage device by the voiceprint information extraction unit and extracting voiceprint information characterizing the respective registered speakers;
The grouping unit calculates a relative distance of voiceprint information of all registered speakers, and a registered speaker having voiceprint information whose relative distance between the registered speakers is within a first threshold. Grouping step,
Setting, by the reference person setting unit, a registered speaker having voiceprint information for which the sum total of relative distances to voiceprint information of other registered speakers belonging to each group is minimum as a reference person in the group; ,
Storing the voiceprint information extracted by the voiceprint information storage unit in the voiceprint database together with the identifier of the registered speaker, the group identifier to which the registered speaker belongs, and the information of the reference person;
Acquiring a call voice of a caller speaker by the voice acquisition unit, and extracting voiceprint information of the caller speaker;
The group selection unit calculates the relative distance only between the voiceprint information of the originating speaker and the voiceprint information of the reference person of each group registered in the voiceprint database, with reference to the voiceprint database. Selecting a group to which a reference person having voiceprint information with the smallest relative distance belongs;
The relative distance between the voiceprint information of the source speaker and the voiceprint information of the registered speaker other than the reference person belonging to the selected group is calculated by the registered speaker selection unit, and this relative distance Calculation is repeated for all registered speakers other than the reference person, and voiceprint information having the smallest relative distance in the selected group is included, and the relative distance is within the second threshold smaller than the first threshold. Selecting a speaker to be registered in
Outputting a warning message including the identifier of the selected registered speaker by the message output unit by the called party talker to the calling speaker so as to be visible or capable of voice recognition during the call;
When the number of registered speakers grouped into one group by the grouping unit exceeds a predetermined number of registered persons by the group reconstruction unit, the group is selected within the group which is selected within the allowable call response interval time. And D. reducing the first threshold to regroup the registered speakers having voiceprint information within the reduced first threshold so that the selection process of the registered speaker in S. is completed. A harmful customer detection method characterized by The voiceprint database further stores together harmful customer rank information indicating the degree of harmful customers of the registered speaker in association with the extracted voiceprint information.
The step executed by the message output is a first warning message if the selected customer's harmful customer rank information of the registered speaker is less than or equal to a predetermined value, and if the harmful customer rank information is greater than the predetermined value. 6. The harmful customer detection method according to claim 5, wherein the warning message of 2 is output respectively. The above harmful customer detection method is further
Storing the telephone installation history information of the user in the telephone number database together with the telephone number of the telephone by the telephone number storage unit;
The second message output unit refers to the telephone number database and executes the credit check of the caller on the basis of the telephone installation history information of the caller telephone number obtained from the caller call information, and the credit check The harmful customer detection method according to claim 5 or 6, further comprising the step of: outputting the result of (4) as a credit check message so as to allow visual recognition or voice recognition by the destination speaker before the start of a call. The above harmful customer detection method is further
The harmful customer detection method according to any one of claims 5 to 7, further comprising the step of: detecting an occurrence of fluctuation of the input call voice by the emotion analysis unit and notifying the message output unit. A harmful customer detection program for causing a computer to execute a harmful customer detection process for detecting in real time whether or not a speaker in a call corresponds to each of all previously registered harmful customers, the program comprising: , To the computer,
Voiceprint information extraction processing for reading out from the storage device a recorded voice signal of a registered speaker identified in advance and extracting voiceprint information characterizing each registered speaker;
A grouping process in which relative distances of voiceprint information between all registered speakers are calculated, and registered speakers having voiceprint information whose relative distances between the registered speakers are all within the first threshold are grouped. When,
A reference person setting process of setting a registered speaker having voiceprint information having a minimum sum of relative distances with voiceprint information of other registered speakers belonging to each group as a reference person in the group;
Voiceprint information storage processing for storing the extracted voiceprint information in the voiceprint database together with the identifier of the registered speaker, the group identifier to which the registered speaker belongs, and the information of the reference person;
Voice acquisition processing of acquiring a call voice of a caller, and extracting voiceprint information of the caller;
The relative distance is calculated only between the voiceprint information of the originating speaker and the voiceprint information of the reference person of each group registered in the voiceprint database with reference to the voiceprint database, and the relative distance is the largest. Group selection processing for selecting a group to which a reference person having small voiceprint information belongs;
The relative distance between the voiceprint information of the source speaker and the voiceprint information of the registered speaker other than the reference person belonging to the selected group is calculated, and the calculation of the relative distance is performed for all subjects other than the reference person. Select a registered speaker having voiceprint information having the smallest relative distance in the selected group, and the relative distance being within a second threshold smaller than the first threshold, repeatedly for the registered speaker Called speaker selection processing,
Message output processing for outputting a warning message including the identifier of the selected speaker to be recognized or voice-recognizable during a call by the called party speaking to the calling party;
When the number of registered speakers grouped into one group in the grouping process exceeds a predetermined number of registered users, the number of registered speakers in the group selected within the allowable call response interval time A computer including: group reconstruction processing for reducing the first threshold to end selection processing, and regrouping a registered speaker having voiceprint information within the reduced first threshold; A harmful customer detection program characterized by being for execution. The voiceprint database further stores together harmful customer rank information indicating the degree of harmful customers of the registered speaker in association with the extracted voiceprint information.
The message output process is configured to generate a first warning message when harmful customer rank information of the selected registered speaker is less than a predetermined value, and a second warning message when the harmful customer rank information is larger than the predetermined value. The harmful customer detection program according to claim 9, wherein: the harmful customer detection program. The above-mentioned harmful customer detection program further
A telephone number storage process of storing telephone installation history information of the user in a telephone number database together with the telephone number of the telephone;
The credit check of the caller is executed based on the telephone installation history information of the caller telephone number obtained from the caller call information with reference to the telephone number database, and the result of the credit check is used as a credit check message. The harmful customer detection program according to claim 9 or 10, further comprising: a second message output process of outputting a voice recognition enabling visual recognition by the called party before the call starts. The above-mentioned harmful customer detection program further
The harmful customer detection program according to any one of claims 9 to 11, further comprising an emotion analysis process of detecting occurrence of fluctuation of the input call voice and notifying a message output unit of the occurrence.